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PREFACE 


Due  to  the  large  number  of  projects  involved,  this  Pictorial  History  of  the  Code  717  Unmanned 
Systems  Group  is  chronologically  presented  in  three  successive  volumes: 

•  Volume  1  (TD  3289)  provides  background  information  on  the  organizational  structure, 
technology  focus  areas,  key  service  areas,  and  facilities,  followed  by  detailed  project 
summaries  for  early  work  over  the  period  1970  through  1999. 

•  Volume  2  (TD  3301)  continues  the  project-summary  discussions  for  those  efforts  begun  in 
2000  through  2009. 

•  Volume  3  (TD  3302)  continues  the  project-summary  discussions  for  those  efforts  begun  in 
2010  through  2016. 

This  is  Volume  I. 
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“Those  who  cannot  remember  the  past  are  condemned  to  repeat  it.” 

George  Santayana 


INTRODUCTION 

From  a  military-application  perspective,  unmanned  systems  often  serve  as  remotely  operated  or 
autonomous  intelligence,  surveillance,  and  reconnaissance  (ISR)  platforms,  which  is  especially 
relevant  for  the  following  reasons: 

•  ISR  is  the  primary  mission  for  small  unmanned  surface  vehicles  (US  Vs)  per  the  Navy 
Unmanned  Surface  Vehicles  (USVs)  Master  Plan  ( USVMP ,  2007). 

•  ISR  is  the  top  priority  mission  according  to  the  OPNAV  N6/N7  Unmanned  Underwater 
Vehicles  (UUVs)  Master  Plan  ( UUVMP ,  2004). 

•  Urban  ISR  is  a  primary  mission  for  small  unmanned  ground  vehicles  (UGVs)  per  the 
Joint  Robotics  Program  Master  Plan  ( JRPMP ,  2004). 

Although  the  Code  717  Unmanned  Systems  Group  has  traditionally  been  more  associated  with 
ground  robotics,  the  full  scope  of  its  involvement  actually  encompasses  all  operational  domains  of 
air,  land,  and  sea,  as  further  discussed  below. 

UNMANNED  AIR  VEHICLES 

The  Unmanned  System  Group’s  involvement  in  the  Unmanned  Air  Vehicle  (UAV)  domain  began 
in  the  early  1980s  in  the  form  of  an  electrically  powered  quad-rotor  platform  intended  to  transport  a 
remote-presence  stereoscopic  vision  system  for  ISR  missions.  This  feasibility  prototype  of  the 
Airborne  Remotely  Operated  Device  (AROD)  was  developed  by  Moller  International  as  a 
subcontractor  to  Perceptronics  for  the  Hawaii  Laboratory  of  the  Naval  Ocean  Systems  Center 
(NOSC),  a  predecessor  organization  of  Space  and  Naval  Warfare  Systems  Center  Pacific  (SSC 
Pacific).  Code  717  has  since  expanded  its  air-domain  involvement  to  include  ducted- fan,  helicopter, 
and  fixed-wing  aircraft,  often  mated  to  supporting  UGVs  (Figure  1). 
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Figure  1.  Pilot  and  flight-electronics  engineer  Nick  Stroumtsos,  holding  a  4-inch  semi-autonomous 
Syma  Sky  Thunder  RC  D63  quadrotor  in  his  right  hand,  is  surrounded  by  the  Unmanned  Systems 
Group’s  growing  fleet  of  unmanned  aerial  vehicles,  circa  March  2016. 
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As  of  January  2016,  our  growing  collection  of  UAV  assets  included  the  following: 

•  One  Allied  Aerospace  iStar  ducted-fan  vertical  takeoff  and  landing  (VTOL)  UAV 

•  One  Guided  Systems  helicopter  with  Piccolo  flight  management  unit 

•  One  spare  airframe  for  a  Guided  Systems  helicopter 

•  One  Guided  Systems  electric  helicopter 

•  One  Senior  Telemaster  (fixed-wing)  UAS  with  Piccolo  flight-management  unit 

•  One  spare  Piccolo  flight-management  unit 

•  One  Arcturus  T-15  fixed-wing  UAV  with  ground  control  station  and  launcher 

•  Three  3DRobotics  Iris  quadrotor  UAVs 

•  One  Applied  Research  Associates  Tactical  Mini  UA  V  (TACMA  V) 

•  Three  AeroVironment  RQ-11A  Raven  A  aircraft  with  two  ground  control  stations  (on  loan 
fromPM-UAS) 

•  Six  AeroVironment  RQ-11B  RavenB  aircraft  with  three  ground  control  stations  (three  on  loan 
from  PM-UAS) 

•  Three  AeroVironment  RQ-20  Puma  UAVs  and  ground  control  stations 

•  One  Aeryon  SkyRanger  UAS  with  ground  control  station  (on  loan  from  U.S.  Pacific  Fleet 
(USPACFLEET) 

•  One  Black  Diamond  MTS  common  ground  control  unit 

•  Two  Honeywell  RQ-16B  T-Hawk  VTOL  micro-air  vehicles  (MAVs)  and  control  station 

UNMANNED  GROUND  VEHICLES 

The  Code  717  Unmanned  Systems  Group  at  SSC  Pacific  has  long  been  involved  in  ground 
robotics  for  two  compelling  reasons: 

1 .  The  Navy’s  joint-service  explosive-ordnance-disposal  (EOD)  charter 

2.  Navy  science  and  technology  support  for  the  United  States  Marine  Corps 

Under  DoD  Directive  5160.62,  the  Department  of  the  Navy  is  formally  assigned  as  executive 
manager  for  all  DoD  RDT&E  involving  EOD,  and  for  decades  robotics  has  been  a  big  part  of  that 
picture.  The  Naval  Surface  Warfare  Center  (NSWC)  Indian  Head  Explosive  Ordnance  Disposal 
Technology  Division  (IHEODTD),  formerly  Naval  Explosive  Ordnance  Disposal  Technology 
Division  (NAVEODTECHDIV),  manages  UGV  acquisition  and  development  for  Joint  EOD 
operations  under  the  auspices  of  the  Naval  Sea  Systems  Command  (NAVSEA).  SSC  Pacific  provides 
technical  support  to  IHEODTD  for  the  man-portable  robots  used  to  neutralize  improvised  explosive 
devices  (IEDs),  the  number-one  threat  faced  by  warfighters  in  Iraq  and  Afghanistan. 

As  of  January  2016,  the  Unmanned  Systems  Group’s  collection  of  teleoperated  man-portable  UGV 
assets  included  the  following: 

•  Five  iRobot  PackBots 

•  Nine  QinetiQ  Talons 

•  Nine  EOD  Performance  Vanguard  robots 

•  One  four-wheel  Segway  robot 

•  Two  tracked  Segway- based  robots 

•  Numerous  smaller  ground  robots 

The  autonomous  ground  vehicles  under  development  onsite  included: 

•  Two  commercial  Rzrs  with  drive-by-wire  kits 

•  One  militarized  Rzr  with  drive-by-wire  kit 
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•  One  Polaris  Ranger  with  QinetiQ  drive-by-wire  kit 

•  One  roboticized  drive-by-wire  Ford  Escape 

•  One  roboticized  High-Mobility  Multi-  Wheeled  Vehicle  ( HMMWV) 

•  One  custom-built  RaDER  UGV 

The  Unmanned  Systems  Group  also  has  a  dozen  or  so  one-of-a-kind  historical  ground  robots  in  the 
museum  area  in  Building  624,  Seaside  (Figure  2). 


Figure  2.  Ground  robots  in  the  museum  area  of  Building  624  (left  to  right  across  the  back  wall) 
include  two  ModBots,  A  Denning  Sentry,  an  MDARS-Interior  security  robot,  ROBART  I,  ROBART II, 
and  ROBART  III.  An  iRobot  All  Terrain  Robotic  Vehicle  is  at  image  center,  with  the  original  prototype 
of  the  URBOT  at  lower  right.  A  few  prototype  ducted-fan  air  vehicles  are  at  foreground  left. 

UNMANNED  SURFACE  VEHICLES 

The  Code  717  Unmanned  Systems  Group  has  an  extensive  history  in  developing  component 
technologies  required  for  robust  USV  operation  in  real-world  environments,  with  a  focus  on 
autonomous  navigation,  obstacle  avoidance,  and  path  planning.  This  work  began  in  2002  with  the 
transition  of  supporting  technologies  developed  under  UGV  programs  funded  by  the  US  Army  and 
Marine  Corps.  After  early  evaluation  of  the  Science  Applications  International  Corporation  (SAIC) 
Owl  Mk  III  platform, 1  the  Code  111  Unmanned  Systems  Group  developed  two  sophisticated  US  Vs 
based  upon  a  jet-drive  Sea  Doo  Challenger  and  an  outboard  Boston  Whaler  (Figure  3).  See 
Unmanned  Surface  Vehicle  in  the  Project  Summaries  section,  Volume  2. 


1  The  Owl  was  developed  by  International  Robotic  Systems  in  1983,  followed  by  a  new-design  Owl  MK  II  in  1992. 
In  1995,  International  Robotic  Systems  became  NAVTEC,  Inc.,  which  later  partnered  with  SAIC  on  the  Owl  Mk  III. 
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Figure  3.  a)  Our  initial  USV  was  based  upon  a  jet-drive  Sea  Doo  Challenger  outfitted  with  GPS, 
video,  lidar,  and  radar  sensors,  b)  Similarly  equipped,  the  second-generation  USV  was  based  on  a 
Boston  Whaler  outboard  drive. 


UNMANNED  UNDERSEA  VEHICLES 

The  Space  and  Naval  Warfare  Systems  Center  Pacific  has  been  developing  and  improving 
remotely-operated  undersea  vehicles  since  1965.  Early  Center  experience  included  development  of 
the  Cable-controlled  Underwater  Recovery  Vehicle  (CURV),2  the  Remote  Unmanned  Work  System 
(RUWS),  the  Mine  Neutralization  Vehicle  (MNV),  and  the  Work  Systems  Package  (Morinaga  & 
Hoffman,  1991).  The  development  and  testing  of  unmanned  underwater  vehicles  (UUVs)  today  falls 
under  the  Unmanned  Maritime  Vehicle  Lab  headed  by  Rich  Arrietta  in  the  Maritime  Systems 
Division  (564),  Bayside. 

Since  1970,  there  has  been  significant  collaboration  between  the  undersea  group  and  the  Advanced 
Systems  Division  (Code  53)3  on  a  number  of  projects  at  the  NOSC  Hawaii  Detachment  at  Kaneohe 
Marine  Corps  Air  Station  on  Oahu.  The  Advanced  Tethered  Vehicle  (ATV)  follow-on  to  RUWS,  for 
example,  was  a  purpose-built  UUV  intended  for  operation  at  depths  as  great  as  20,000  feet.  Initiated 
in  1988,  the  Teleoperator  Telepresence  System  /  Concept  Verification  Model  (TOPS/CVM)  was  a 
highly  dexterous  telerobotic  master-slave  work  system  that  could  mimic  the  manipulative  dexterity 
of  humans.  Both  these  and  other  joint  UUV  efforts  will  be  further  discussed  in  the  chronological 
Project  Summaries  section  of  this  volume. 

Other  remotely  operated  and  autonomous  development  efforts  managed  by  the  Unmanned 
Maritime  Vehicle  Lab  included: 

•  Advanced  Unmanned  Search  System  (A  USS) 

•  Distributed  Surveillance  Sensor  Network  (DSSN) 

•  Flying  Plug 

•  Free  Swimming  Vehicle 

•  Submersible  Cable-Actuated  Teleoperator  (SCAT) 

•  Snoopy 

•  Solid  Rocket  Booster  Nozzle  Plug  (SRB/NP) 

See  http://www.public.navy.mil/spawar/Pacific/Robotics/Pages/Underwater  Vehicles.aspx. 


2  CURV  was  used  to  recover  the  hydrogen  bomb  lost  off  Palomares,  Spain,  in  1966;  the  third-generation  CURV  III 
was  used  to  attach  a  recovery  line  and  pull  the  two-man  crew  of  the  bottomed  PISCES  III  submersible  to  safety  off 
Cork,  Ireland  in  1973  (Christ  &  Wemli,  2013). 

3  The  Advanced  Systems  Division  (Code  53)  was  the  forerunner  of  the  current  Advanced  Systems  and  Applied 
Sciences  Division,  Code  71700. 
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BACKGROUND  AND  MISSION 


With  over  35  years  of  hands-on  experience,  the  Code  717  Unmanned  Systems  Group  at  SSC 
Pacific  has  developed  core  in-house  expertise  in  unmanned- vehicle  technologies  and  attained  a 
recognized  leadership  role  in  their  design,  development,  fielding,  and  support.  As  depicted  in  Figure 
4,  the  Unmanned  Systems  Group’s  fundamental  role  is  to  serve  as  an  enabling  interface  between  the 
development  and  DoD  user  communities  of  unmanned  systems.  From  primary  involvement  with 
United  States  Marine  Corps  (USMC)  and  Army  unmanned  ground  vehicles  some  30  years  ago,  the 
Center  has  since  led  numerous  S&T  efforts  for  DoD  with  original  research,  applied  research,  and 
technology  application  on  a  wide  variety  of  robotic  platforms  and  missions.  Within  the  last  10  years, 
the  number  and  types  of  systems  have  increased  greatly  in  all  operational  domains  of  air,  land,  and 
sea. 


Figure  4.  The  primary  role  of  the  Code  717  Unmanned  Systems  Group  at  SSC  Pacific  is  to  serve  as 
the  “impedance-matching  transformer”  between  unmanned-systems  users  and  the  supporting 
technical  communities,  two  codependent  groups  that  typically  do  not  speak  the  same  language. 

Complementing  the  unmanned-vehicle  development  work  (air,  land,  and  sea),  we  are  equally 
immersed  in  all  aspects  of  the  associated  command,  control,  and  communication  needs,  as  will  be 
further  discussed  under  various  efforts  in  the  Project  Summaries  section.  In  addition  to  development, 
rapid  prototyping,  and  testing,  the  Code  717  Unmanned  Systems  Group  also  provides  quick-response 
incident  and  in-theater  support  when  needed: 

•  Immediately  after  9/1 1,  we  were  tasked  by  the  Robotic  Systems  Joint  Program  Office  (RS 
JPO)  to  deploy  a  robotic  search-and-rescue  team  to  the  World  Trade  Center. 

•  Just  6  months  later,  we  provided  three  in-house-developed  man-portable  robots  to  Navy  EOD 
Mobile  Unit  3  for  deployment  to  Afghanistan  during  Operation  Enduring  Freedom. 

•  Developed  that  same  year  (2002)  for  the  US  Army  Chemical  School,  the  prototype  Chemical 
and  Hazardous  Avoidance  Robotic  System  (CHARS)  payload  was  ported  to  four  iRobot 
PackBot  Explorers  provided  via  the  RS  JPO  to  the  82nd  Airborne  Division  during  Operation 
Iraqi  Freedom  in  2003. 
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•  From  2004  to  2007,  the  Center  deployed  two-man  Unmanned  Systems  Reserve  Unit  teams 
on  recurring  6-month  rotations  to  staff  up  the  RS  JPO  Joint  Robotics  Repair  Facility  in  Camp 
Victory,  Iraq. 

ORGANIZATIONAL  STRUCTURE 

The  Center’s  fledgling  unmanned  systems  program  reached  critical  mass  in  1986  through  the 
synergistic  confluence  of  three  geographically  dispersed  groups: 

•  The  NAVSEA  Office  of  Robotics  and  Autonomous  Systems  (SEA  90G),  Naval  Sea  Systems 
Command,  Washington,  DC 

•  The  Advanced  Systems  Division  (Code  53),  Naval  Ocean  Systems  Center  (NOSC), 

San  Diego,  CA 

•  The  Autonomous  Systems  Branch  (Code  442),  NOSC  Hawaii  Laboratory,  Kaneohe  Marine 
Corps  Air  Station,  Oahu,  HI 

The  contributing  roles  played  by  each  of  these  pioneering  organizations  are  more  fully  described  in 
the  following  subsections. 

NAVSEA  Office  of  Robotics  and  Autonomous  Systems  (SEA-90G) 

In  summer  1982,  Vice  Admiral  Earl  B.  Fowler,  Commander,  Naval  Sea  Systems  Command 
(NAVSEA),  established  a  billet  for  the  Special  Assistant  for  Robotics  within  the  NAVSEA 
Acquisition,  Planning  and  Appraisal  Directorate  (SEA  90).  Following  completion  of  his  thesis  at  the 
Naval  Postgraduate  School  in  Monterey,  CA  (Figure  5),  engineering  duty  officer  (EDO)  Lieutenant 
Commander  H.R.  (Bart)  Everett  was  assigned  to  this  billet  in  November  that  same  year  as  SEA 
90M3  (Appendix  A).  The  overarching  intent  was  to  identify  and  coordinate  all  NAVSEA  efforts  in 
robotics  to  minimize  redundancy,  ensure  compatibility  of  independently  developed  systems,  and 
avoid  the  risks  of  inappropriately  assigned  or  ill-conceived  applications  (Everett,  1984). 


Figure  5.  ROBART I  (lower  left),  Lieutenant  Commander  Bart  Everett’s  thesis  project  at  the  Naval 
Postgraduate  School  in  Monterey,  CA,  triggered  a  media  frenzy  that  led  to  Everett’s  1982 
assignment  as  Special  Assistant  for  Robotics  to  Vice  Admiral  Earl  Fowler,  Commander,  Naval  Sea 
Systems  Command  (Appendix  C). 
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To  facilitate  this  endeavor.  Lieutenant  Commander  Everett  recruited  a  diverse  team  from  two 
sources:  1)  NAVSEA  program  managers  interested  in  the  application  of  robotic  technology,  and 
2)  evolving  subject  matter  experts  from  the  supporting  Navy  Laboratories.  What  would  later  be 
referred  to  as  an  Integrated  Product  Team  (IPT),  this  group  came  to  be  known  at  the  time  as  the 
NAVSEA  Robotics  Council.  As  the  NAVSEA  Integrated  Robotics  Program  grew  in  scope  (Figure 
6),  the  Office  of  Robotics  and  Autonomous  Systems  (SEA  90G)  was  officially  stood  up  under 
Lieutenant  Commander  Everett  in  May  1984  (Appendix  B),  with  70  ongoing  efforts  catalogued  by 
end  of  the  following  year  (Everett,  1985a).  In  recognition  of  the  increased  oversight  requirements, 
former  naval  officer  William  Butler  was  hired  to  assist  as  SEA  90G1 . 


Figure  6.  Program  goals  and  objectives  for  the  NAVSEA  Integrated  Robotics  Program  managed  by 
the  Office  of  Robotics  and  Autonomous  Systems  (SEA  90G),  Naval  Sea  Systems  Command, 
Washington,  DC  (adapted  from  Everett,  1985a). 

To  provide  corporate  awareness  of  all  DoD  development  efforts,  the  NAVSEA  Robotics  and 
Artificial  Intelligence  Database  (RAID)  was  established  by  SEA  90G  (Figure  7),  to  be  maintained  by 
Code  9302  at  NOSC  in  San  Diego,  CA.  Accessible  to  approved  DoD  users  via  the  ARPANET  after 
coming  online  in  1984, 4  this  database  supported  three  broad  categories  of  information: 

•  The  Project  Information  section  identified  the  principal  investigator,  sponsor,  technical 
synopsis,  and  location  of  the  performing  research/development  activity. 


4  Created  by  the  DoD  Advanced  Research  Projects  Agency  (ARP A),  now  the  Defense  Advanced  Research  Projects 
Agency  (DARPA),  the  ARPANET  was  the  precursor  to  today’s  internet. 
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•  The  Contacts  Information  section  included  the  mailing  addresses,  phone  numbers,  special 
interest  areas,  and  e-mail  addresses  of  people  and  organizations  in  the  field. 

•  The  Bibliographic  References  section  contained  authors,  titles,  and  short  abstracts  of 
pertinent  literature  from  Department  of  Defense  (DoD),  industry,  and  academia. 


DATA  SOURCES 


ORGANIZATION 


ACTIVE  RAID  USERS 

(PRESENTLY  97  PERSONNEL) 


>OUSDRE 

>  ARMY  -  HEL.  FSTC  AMC,  ARI 
TACOM,  ARO,  ETC. 

AND  CONTACTORS 

>NAVY  -  OPNAV,  NSEA  ONR 

NAVAIR,  SPAWAR.  NADC, 
NARF,  NOSC,  NPS,  NRL. 
ONT,  USNA,  ETC.,  AND 
CONTACTORS 

'  DARPA  -  AND  CONTRACTORS 

►  USAF  -  AFALC,  AFWAL.  AFLC 
AND  CONTRACTORS 

i  DoC 


DoE,  SANDIA,  ARGONNE,  OAKRIDGE 
AND  CONTRACTORS 

NASA  -  AMES,  GODDARD, 

MARSHALL.  KENNEDY 

OTHER  DoD  DATABASES  -  MTIAC, 
SIMON,  MANTIS 


Figure  7.  RAID  was  installed  on  a  VAX-1 1/780  computer  at  NOSC,  chosen  for  this  role  by  SEA  90G 
because  of  its  experience  in  robotics,  telecommunications,  and  database  management  (adapted 
from  Everett,  1985a). 

Over  700  projects  had  been  entered  in  RAID  by  the  close  of  1985,  with  97  registered  users  from 
some  50  organizations  (Everett,  1985a).  By  this  point,  SEA  90G  had  developed  a  structured 
assessment  methodology  that  was  employed  by  the  NAVSEA  Robotics  Council  to  break  down 
targeted  application  areas  into  associated  technology  needs,  which  were  catalogued  as  follows: 

•  Those  component  technologies  known  to  exist. 

•  Ongoing  technology  developments  likely  to  become  available  in  the  near  term. 

•  Unaddressed  technology  needs  relatively  unique  to  the  project  application. 

The  RAID  database  provided  the  enabling  mechanism  for  this  assessment  and  subsequent 
documentation  of  the  results.5 

A  secondary  benefit  of  this  process  was  the  inherent  generation  of  a  comprehensive  listing  of 
identified  technology  shortfalls  common  to  multiple  ongoing  NAVSEA  efforts  (Everett,  1985b). 
These  findings  were  coordinated  with  and  endorsed  by  Dr.  Randy  Shumaker,  Director  of  the  Navy 
Center  for  Applied  Research  in  Artificial  Intelligence  (NCARAI),  part  of  the  Information 
Technology  Division  within  the  Naval  Research  Laboratory.  Joint  NAVSEA/NCARAI 
recommendations  to  address  these  concerns  were  subsequently  briefed  to  the  Office  of  Naval 


5  Following  Everett’s  transfer  to  NOSC  in  1986,  oversight  of  the  RAID  database  was  handed  off  to  Rome  Air 
Development  Center,  NY,  where  it  eventually  languished  and  was  ultimately  abandoned. 
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Research  (ONR)  and  Office  of  Naval  Technology  (ONT)  on  6  September  1985  (Appendix  B).  At  the 
suggestion  of  Vice  Admiral  Fowler,  NOSC  began  a  focused  initiative  to  further  mitigate  these 
technology-base  deficiencies,  with  Lieutenant  Commander  Everett  flag-detailed  to  the  Center  to 
oversee  this  effort  in  November  1986  (Appendix  D). 

Prior  to  his  transfer  to  San  Diego,  Everett  was  visited  by  Dr.  Will  Rasmussen  (Code  44)  and  Dr. 
Doug  Gage  of  NOSC,  who  recruited  him  into  the  Autonomous  Systems  Branch  (Code  442).  In  a 
follow-up  visit,  Dr.  John  Silva  (who  headed  the  Center’s  Independent  Exploratory  Development 
(IED)  program  (Code  014),  offered  to  fund  a  3-year  IED  project  entitled  Modeling  the  Environment 
of  a  Mobile  Security  Robot ,  provided  that  ROB  ART  II  be  made  available  as  a  supporting  laboratory 
surrogate.6  This  small  in-house  effort  served  as  the  springboard  for  ultimately  establishing  the 
Autonomous  Systems  Branch  (reorganized  as  Code  535)  at  NOSC  as  the  foremost  DoD  subject 
matter  experts  in  robotics  for  physical  security  (Babb,  1990). 

NOSC  Autonomous  Systems  Branch  (Code  442) 

The  Naval  Ocean  Systems  Center  in  San  Diego,  CA,  was  one  of  four  Navy  Laboratories 
supporting  the  NAVSEA  Office  of  Robotics  and  Autonomous  Systems  (SEA  90G)  in  the  mid-1980s 
timeframe.  As  previously  mentioned,  Code  9302  at  NOSC  oversaw  implementation,  data  entry,  and 
technical  management  of  the  NAVSEA  Robotics  and  Artificial  Intelligence  Database  (RAID).  The 
Autonomous  Systems  Branch  (Code  442),  which  supported  SEA  90G  via  the  NAVSEA  Robotics 
Council,  was  developing  a  weld  process-control  expert  system.  This  work  was  funded  by  the 
Manufacturing  Technology  Program,  Naval  Material  Command  (NAVMAT),  as  part  of  the  SEA- 
070 A  Integrated  Flexible  Welding  System  (IFWS)  project  (Everett,  1985c). 

Code  442  was  also  funded  by  the  USMC  Exploratory  Development  (6.2)  Surveillance  Program  to 
develop  the  Ground  Surveillance  Robot  (GSR)  shown  in  Figure  8  (Harmon,  1984).  The  GSR  project 
explored  the  design  of  a  modular,  flexible,  distributed  architecture  for  the  integration  and  control  of 
complex  robotic  systems,  using  a  fully  actuated  7-ton  M-114  Command  and  Reconnaissance  Carrier 
as  the  testbed  host  vehicle.  With  an  array  of  ultrasonic  sensors  (fixed  and  steerable)  and  a  distributed 
blackboard  architecture  implemented  on  multiple  PCs,  the  vehicle  successfully  demonstrated 
autonomous  following  of  both  a  lead  vehicle  and  a  walking  human  in  1986  (Harmon,  1987).  Shortly 
after  Everett’s  arrival  in  San  Diego,  the  Autonomous  Systems  Branch  (Code  442)  was  reorganized 
under  Code  53  as  the  Autonomous  Systems  Branch  (Code  531). 


6  ROBART II  was  a  second-generation  security  robot  constructed  by  Everett  in  the  basement  of  his  home  in 
Springfield,  VA,  as  follow-on  to  ROBART /,  his  thesis  project  at  the  Naval  Postgraduate  School  (NPS)  in  Monterey, 
CA.  Both  of  these  systems  are  further  discussed  in  the  Project  Summaries  section  of  this  volume. 
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Figure  8.  The  Code  442  autonomous  Ground  Surveillance  Robot  (GSR)  was  based  upon  an  M-114 
Command  and  Reconnaissance  Carrier.  At  right  is  ROBART II,  newly  arrived  from  Springfield,  VA, 
circa  December  1986. 

Advanced  Systems  Division  (Code  53) 

Located  on  the  northeast  shore  of  Oahu  (Figure  9),  the  Advanced  Systems  Division  (Code  53)  of 
the  NOSC  Hawaii  Laboratory  played  an  early  role  in  the  development  of  unmanned  systems  for 
Navy  and  USMC  customers  (Table  1).  Initial  pursuit  of  underwater  applications  such  as  the  Remote 
Unmanned  Work  System  (RUWS)  began  in  the  early  1970s  (see  later  discussion  under  chronological 
Project  Summaries).  In  both  the  undersea  and  ground  robotics  domains,  a  key  focus  became 
teleoperated  remote-presence  systems  (binaural  audio  and  stereo-vision  feedback  via  helmet- 
mounted  displays)  for  reducing  the  control  burden  imposed  upon  human  operators. 
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Figure  9.  The  NOSC  Hawaii  Lab  (lower  left)  was  scenically  situated  just  south  of  the  threshold  of 
Runway  22  at  Marine  Corps  Air  Station  Kaneohe  (now  Marine  Corps  Base  Hawaii)  on  the  northeast 
side  of  Oahu,  HI. 

Table  1 .  Code  53  branches  and  branch  heads  at  the  NOSC  Hawaii  Lab  as  of  January  1988.  Code 
535,  the  Advanced  Technology  Development  Branch,  was  stood  up  and  located  in  San  Diego. 


Code 

Branch  Name 

Branch  Head 

531 

Electrical  Systems  Branch 

D.C.  Smith 

532 

Mechanical  and  Teleoperator  Branch 

T.W.  Hughes 

533 

Cognitive  Sciences  Branch 

J.K.  Katayama 

534 

Advanced  Fiber  Optic  Systems  Branch 

A.T.  Nakagawa 

Early  investigation  and  development  of  this  technology  began  in  the  late  1970s  (Figure  10a), 
continuing  under  the  Remote  Presence  Demonstration  System  (RPDS)  project  in  1983  using  the 
anthropomorphic  master-slave  configuration  shown  in  Figure  10b.  An  instrumented  exoskeleton 
worn  by  the  operator  generated  servo-commands  that  caused  the  hydraulically  actuated  “Green  Man” 
slave  to  replicate  the  same  motions  (Figure  11a).  This  master-slave  control  strategy  would  also 
mimic  head  movements,  with  two  cameras  attached  to  the  robot’s  head  providing  stereo  imagery  to  a 
pair  of  video-camera  eyepiece  monitors  mounted  on  an  aviator's  helmet  worn  by  the  operator. 
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Figure  10.  a)  This  early-70s  telepresence  demonstration  platform  was  based  upon  a  small  tracked 
ground  robot  (bottom  left),  with  a  preliminary  helmet-mounted  stereo  display  worn  by  the  operator, 
b)  This  anthropomorphic  “Green  Man”  configuration,  circa1981 ,  was  outfitted  with  hydraulic  arms, 
manipulators,  and  a  moveable  head. 


This  evolving  remote-presence  technology  was  especially  relevant  to  certain  ground  applications 
of  interest  to  the  US  Marine  Corps.  The  RPDS  prototype  shown  in  Figure  lib  was  constructed  by  the 
Hawaii  Lab  to  demonstrate  how  such  a  system  could  provide  natural  and  intuitive  control  of 
teleoperated  ground  vehicles.  Hosting  an  articulated  dual-camera  system  with  an  associated  helmet- 
mounted  display,  this  small  surrogate  vehicle  provided  the  operator  with  both  stereo  vision  and 
binaural  hearing,  the  pan-tilt  actuators  being  slaved  to  the  operator’s  head  motion  and  vehicle 
steering  controls.  There  was  also  a  microphone/speaker  system  to  project  the  operator’s  voice  from 
an  on-board  speaker  via  the  multi-conductor  electrical  tether  (Umeda,  2015). 


Figure  1 1 .  a)  Kirk  Jennings  operates  an  improved  “Green  Man”  master-slave  system  developed 
under  the  Remote  Presence  Demonstration  System  (RPDS)  project,  b).  The  elevated  ball  atop  the 
front  of  this  small  indoor  robot  operated  by  Dave  Smith  provided  a  vehicle-based  visual  reference  for 
the  stereo  camera  pair,  circa  1982. 
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In  the  early  1980s,  two  reconnaissance,  surveillance,  and  target-acquisition  (RSTA)  projects  were 
undertaken  by  NOSC  under  the  auspices  of  the  US  Marine  Corps  Exploratory  Development  (6.2) 
Surveillance  Program:  1)  the  Advanced  Teleoperator  Technology  (ATT)  effort  by  Codes  531  and  532 
at  the  Hawaii  Lab;  and,  2)  the  previously  mentioned  Ground  Surveillance  Robot  (GSR)  by  Code  442 
in  San  Diego.  Under  the  first  of  these,  the  remote-presence  technology  originally  developed  for  the 
NAVSEA  RUWS  project  was  ported  to  a  teleoperated  dune  buggy.  A  parallel  air- vehicle  concept 
involved  the  vertical-take-off  Airborne  Remotely  Operated  Device  (AROD),  initially  based  on  a 
quadrotor  manufactured  by  Moller  International  (Figure  12). 


Figure  12.  The  tethered  Airborne  Remotely  Operated  Device  (AROD)  was  an  electrically  powered 
quadrotor  developed  by  Moller  International  and  tested  as  an  airborne  remote-presence  system  at 
the  NOSC  HI  Lab. 

The  operator  control  station  for  the  dune-buggy  portion  of  the  ATT  thrust  employed  a  mechanical 
tracking  capability  to  measure  the  position  and  orientation  of  the  driver’s  helmet  (Figure  13a),  which 
was  later  replaced  by  a  non-contact  inductive  system  (Figure  13b).  This  3D-pose  data  was  used  to 
control  the  robotic  pan-and-tilt  unit  in  the  driver’s  seat  of  the  dune  buggy  (Figure  13c),  which 
consequently  mimicked  the  head  motions  of  the  operator.  A  stereo-camera  pair  mounted  atop  this 
pan-and-tilt  unit  thus  looked  in  the  same  direction  and  elevation  as  the  remote  operator  at  the  control 
station.  As  the  focus  was  on  high-quality  telepresence  to  facilitate  remote  driving  under  degraded 
visual  conditions,  a  fiber-optic  tether  was  used  to  provide  high-bandwidth  communications  with 
minimal  latency.7 


7  Support  for  this  component  was  provided  by  the  Advanced  Fiber  Optic  Systems  Branch  (Code  534). 
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Figure  13.  a)  A  mechanical  tracking  system  (out  of  image  left)  attached  to  the  white  helmet 
measured  the  orientation  of  the  operator’s  head,  b)  A  Polhemus  3D  tracking  system  has  replaced 
the  mechanical  version.  The  displays  at  upper  right  provided  visitors  wearing  polarized  glasses  with 
stereo  vision,  c)  The  robot’s  head  mimicked  the  pose  of  the  operator’s  helmet  while  its  cameras 
relayed  stereoscopic  video  to  the  helmet-mounted  display. 

A  secondary  project  goal  was  to  demonstrate  practical  applications  of  fiber  optics  in  general,  and 
Code  534’s  ability  to  design  bidirectional  telemetry  systems  with  data  rates  up  to  200  megabits  per 
second.  Challenges  in  this  UGV  application  included  the  necessary  development  of  cable  winding 
(Figure  14),  payout,  and  recovery  systems,  as  well  as  maintaining  reliable  communications  in  high- 
vibration  environments.  With  regard  to  this  latter  issue,  a  custom-fabricated  fiber-optic  cable  for 
UGV  applications,  ruggedized  with  a  polyurethane  and  Kevlar  jacket,  was  designed  and  ordered  for 
improved  survivability  under  dynamic  real-word  conditions.8 


Figure  14.  Warren  Hahn  and  Clifford  Horikawa  of  the  NOSC  Hawaii  Laboratory  prepare  a  new  spool 
of  fiber-optic  cable  (image  center)  with  a  custom  designed  winding  machine  for  this  purpose. 
Effective  cable  winding,  payout,  and  recovery  were  essential  for  reliable  teleoperation  in  the  field. 


8  The  NOSC  Hawaii  Lab  began  using  fiber-optic  tethers  on  UUVs  in  the  late  1970s  (Nakagawa  &  Smith,  1980). 
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The  exploratory  remote-presence  development  conducted  on  the  dune-buggy  UGVs  ultimately 
transitioned  to  the  USMC  HMMWV- based  Ground/Air  TeleRobotic  System  (GATERS)  prototype 
shown  in  Figure  15.  A  Polhemus  3D  tracker  replaced  the  mechanical  version  employed  earlier  under 
the  dune-buggy  effort  to  measure  the  human  driver’s  head  movements  in  real  time.  The  3-axis 
inductive  sensor  component  mounted  on  the  L-shaped  support  above  the  portable  operator  control 
station  tracked  the  orientation  of  a  3-axis  inductor  mounted  on  top  of  the  driver’s  helmet.  This  non- 
invasive  approach  was  more  accurate  and  far  less  disconcerting  to  the  human  operator. 


Figure  15.  This  early  version  of  the  GATERS  UGV,  shown  prior  to  installation  of  the  Surveillance  and 
Weapon  Modules,  employed  a  Polhemus  3D  tracking  system  (see  blue  sensor  on  L-shaped  support 
above  operator’s  helmet  below).  The  equipment  at  lower  right  comprised  a  portable  operator  control 
station. 


An  embedded  version  of  the  operator  control  station  is  shown  in  Figure  16a,  with  the  blue 
Polhemus  sensor  attached  to  a  wooden  support  above  the  driver’s  head.  GATERS  team  members 
(from  left  to  right)  Scott  McArthur,  Tony  Koyamatsu,  Captain  George  Murray  (USMC),  Derrick 
Kusuda,  Tom  Hughes,  Celia  Metz  (unknown);  Tracy  Heath  (unknown),  and  Alan  Umeda  pose  with 
the  UGV  in  Figure  16b.  Just  left  of  Captain  Murray,  the  stereo  cameras  on  the  robotic  head  have 
been  enclosed  in  a  watertight  fiberglass  enclosure  for  more  rugged  outdoor  testing  in  San  Diego,  as 
further  discussed  under  the  Project  Summaries  section  in  this  volume. 
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b) 

Figure  16.  a)  Early  GATERS  control  station  employing  helmet-mounted  display,  b)  The  GATERS 
UGV  (background)  was  a  descendent  of  the  exploratory  ATT  dune  buggy.  Celia  Metz  (foreground 
center)  was  later  selected  as  Code  53  Division  Head,  while  Tracy  Heath  to  her  left  would  become 
the  Code  71710  branch  head  in  2013. 

The  GATERS  project,  renamed  the  Teleoperated  Vehicle  Project  (TOV)  project,  transitioned  in 
1988  to  Code  535  in  San  Diego  for  system  hardening  and  support  during  field  evaluation  by  the 
Marine  Corps  Tactical  Systems  Support  Activity  (MCTSSA),  Camp  Pendleton,  CA.  Hawaii  Codes 
531,  532,  and  534  continued  to  provide  valuable  technical  assistance  during  this  effort  through  1989. 
Closure  of  the  NOSC  Hawaii  Laboratory,  which  had  been  recommended  under  the  Base  Closure  and 
Realignment  Act  of 1990,  was  unfortunately  approved  in  1991.  Several  of  the  Code  53  projects 
briefly  mentioned  in  this  section  are  discussed  in  greater  detail  in  the  Project  Summaries  section. 

NOSC  Autonomous  Systems  Branch  (Code  535) 

Seriously  outshadowed  by  the  Defense  Advanced  Research  Projects  Agency  (DARPA) 
Autonomous  Land  Vehicle  (ALV)  program,9  all  further  work  on  the  Ground  Surveillance  Robot 
(GSR)  was  cancelled  by  the  USMC  in  1986,  whereupon  the  M-114- based  vehicle  became  an  air-to- 
ground  target  on  San  Clemente  Island.  When  Lieutenant  Commander  Everett  reported  aboard  NOSC 
in  November  of  that  year,  much  of  Code  442  had  already  left  to  work  in  other  branches  or  the  private 
sector  (Figure  17).  Down  to  just  five  members,  the  Autonomous  Systems  Branch  (Code  442)  was 
soon  reorganized  under  the  Advanced  Systems  Division  (Code  53)  as  Code  535. 10 

The  new  business  strategy  was  to  springboard  off  the  highly  successful  Independent  Exploratory 
Development  ROBART //project  to  attract  outside  sponsors  (Babb,  1990).  Commander  Everett  was 
subsequently  contacted  by  the  Armament  Research  Development  and  Engineering  Center  (ARDEC) 
at  Piccatinny  Arsenal  regarding  their  new-start  effort  for  the  U.S.  Army  Physical  Security  Equipment 
Management  Office  (PSEMO)  at  Fort  Belvoir,*  1 1  the  Mobile  Detection  Assessment  Response  System 
(MDARS). 


9  The  AL V project  was  part  of  DARPA’s  Strategic  Computing  Initiative,  which  began  in  1 983  (DARPA,  1 986). 

10  Lieutenant  Commander  Everett  was  offered  the  position  of  Officer  in  Charge  of  the  NOSC  Hawaii  Lab  by  Code 
53  Division  Head  Don  Moore,  but  opted  to  remain  in  San  Diego  and  pursue  his  NAVSEA  tasking. 

11  Now  Product  Manager,  Force  Protection  Systems  (PdM-FPS)  at  Fort  Belvoir,  VA. 
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Figure  17.  Code  442  personnel  in  March  1987,  clockwise  from  upper  left:  Will  Gex,  Gary  Gilbreath, 
Guy  Bianchini,  Walt  Aviles,  Robin  Laird,  Manuel  Solorzano,  Telford  Quon,  Nancy  Campbell, 
Margaret  Myers,  Lieutenant  Commander  Bart  Everett,  Brian  Pinz,  and  Doug  Gage.  Seven  of  the 
above  personnel  would  leave  over  the  next  several  months. 

In  1989,  PSEMO  began  funding  the  Branch  to  support  a  number  of  robotic-security  programs  that 
continued  to  evolve  over  the  next  25  years  (Babb,  1990).  As  this  role  expanded,  Commander  Everett 
eventually  became  the  overall  MDARS  Technical  Director  in  1993,  retiring  from  active  duty  in 
September  of  that  same  year.  Following  a  Federal  hiring  freeze,  Everett  was  rehired  as  Associate 
Division  Head  for  Robotics  (Code  5305)  under  Division  Head  Doug  Murphy  (Code  53)  in  January 
1994. 

In  1996,  the  Adaptive  Systems  Branch  (Code  531)  was  redesignated  as  the  Autonomous  Systems 
Branch  (Code  D371),  Advanced  Systems  Division  (Code  D37),  Space  and  Naval  Warfare  Systems 
Center  San  Diego  (see  Appendix  B).  With  the  retirement  of  Doug  Murphy  in  1999,  Celia  Metz  took 
over  as  Code  D37  Division  Head.  Our  primary  focus  during  this  timeframe  was  on  the  MDARS 
Interior  and  Exterior  programs,  plus  a  new  start  Man  Portable  Robotic  System  (MPRS)  project 
funded  by  NAVSEA.  In  2001,  at  the  request  of  the  RS  JPO  in  Huntsville,  AL,  three  Urban  Robots 
(URBOTs)  developed  under  the  MPRS  project  were  sent  to  the  WTC  site  to  assist  in  search  and 
rescue  operations  (Figure  18). 
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Figure  18.  Bart  Everett  (Code  D3705  Associate  Division  Head  for  Robotics)  and  Robin  Laird  (Code 
D371  Branch  Head)  with  nine  shipping  containers  containing  three  URBOTS  and  associated  support 
equipment  on  the  ground  at  Stewart  Air  National  Guard  Base  in  Newburg,  New  York.  (See  Man 
Portable  Robotic  System  in  Project  Summaries  section.) 

The  Adaptive  Systems  Branch  (Code  D371)  was  reorganized  as  the  Robotic  Systems  Branch 
(Code  2371)  in  early  2002,  with  now  over  ten  funded  projects  in  the  air,  land,  and  sea  domains 
(Figure  19).  The  MDARS Interior  program  was  undergoing  Limited  User  Testing  (LUT)  at 
Susquehanna  Army  Depot,  PA,  while  the  parallel  Exterior  program  had  entered  the  System 
Development  and  Demonstration  phase.  The  Unmanned  Surface  Vehicle  (USV)  project  had  just 
kicked  off  as  an  MDARS  spinoff,  with  preliminary  evaluation  of  the  radio-controlled  SAIC  Owl  Mk 
III  jet-ski.  A  Cooperative  Research  and  Development  Agreement  (CRADA)  had  been  signed  with 
Allied  Aerospace  to  integrate  their  VTOL  iStar  UAV  with  an  MDARS  (BAA)  robot. 


Figure  19.  Members  of  the  Robotic  Systems  Branch  pose  next  to  newly  erected  Building  618  in  the 
F-36  parking  lot  with  representative  robots  from  seven  of  its  ten  ongoing  projects,  circa  March  2002. 
Note  the  29-inch  iStar  UAV  between  the  SAIC  Owl  Mk  III  (foreground  center)  and  the  MDARS  BAA 
ground  robot  (foreground  right). 
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On  19  July  2002,  former  Speaker  of  the  House  Newt  Gingrich  visited  the  Center  on  a  fact-finding 
mission  for  Secretary  of  Defense  Donald  Rumsfeld,  and  spent  considerable  time  with  the  Robotic 
Systems  Branch  (Code  2371).  His  objective  was  to  investigate  promising  technologies  that  could 
potentially  lead  to  significant  operational  advantages  for  warfighters  in  theater.  One  of  the  first 
demonstrations  along  these  lines  was  the  MPRS  URBOT,  which  had  deployed  to  both  the  World 
Trade  Center  (Mullens,  2001)  and  Operation  Enduring  Freedom  (Mullens,  2002a),  with  special 
focus  on  the  new  marsupial-carrier  payload  for  traditional  and  unmanned  vehicles  (Figure  20). 


Figure  20.  Bart  Everett  explains  the  MDARS  (BAA  version)  less-lethal  pepper-ball  gun  pod  (upper 
left)  and  rear-mounted  UGV  marsupial  carrier  for  the  man-portable  URBOT  to  Mr.  Gingrich  in  the 
F-36  parking  lot  Seaside. 

Another  key  event  was  a  free-flight  demonstration  of  the  Allied  Aerospace  (formerly  Micro  Craft) 
iSTAR  29-inch  lift-augmented  ducted-fan  UAV  at  Code  2371 ’s  Central  Test  Site  on  Woodward  Road 
(Figure  21a).  On  14  March  2002,  the  first  known  launch  of  a  VTOL  UAV  from  a  host  UGV  had  been 
conducted  under  a  CRADA  with  Allied  Aerospace  at  the  Holtville  Regional  Airport  just  east  of  El 
Centro,  CA  (Mullens,  2002b).  Under  the  new-start  Autonomous  UAV  Mission  System  (AUMS) 
project,  the  MDARS  UGV  would  soon  be  equipped  with  a  payload  for  autonomous,  launch,  recovery, 
and  refueling  of  VTOL  UAVS,  as  further  discussed  under  Project  Summaries  in  Volume  2  (Mullens, 
2002c). 


19 


Figure  21 .  a)  A  free-flight  demonstration  of  the  Allied  Aerospace  iStar  UAV.  b)  ONR  summer  intern 
Rachel  TenWolde  talks  with  Newt  Gingrich  during  his  visit  to  the  Robotic  Systems  Branch  in 
September  2002.  A  Lean-Six-Sigma  Black  Belt,  Ms.  TenWolde  was  hired  in  2008  and  selected  as 
the  Code  71720  Branch  Head  in  2013. 

During  his  visit,  Mr.  Gingrich  expressed  considerable  interest  in  countermine  and  humanitarian¬ 
demining  applications  of  ground  robots  (Figure  22).  Beginning  in  1994,  the  Center  had  played  a 
supporting  role  in  the  Basic  UXO  Gathering  System  (BUGS)  project  managed  by  the  Naval  EOD 
Technology  Division  (Gage,  1995a).  The  BUGS  objective  was  to  design,  test,  evaluate,  and 
demonstrate  the  use  of  distributed  robotics  in  clearing  unexploded  submunitions  and  minefield 
neutralization  (DeBolt,  O’Donnell,  Freed,  &  Nguyen,  1997).  With  help  from  a  number  of  established 
EOD  players  in  the  field,  Bart  Everett  prepared  a  comprehensive  discussion  of  robotic  countermine 
opportunities,  which  was  hand  delivered  to  Mr.  Gingrich  in  Washington,  DC,  by  Commanding 
Officer  Captain  Tim  Flynn. 


Figure  22.  Newt  Gingrich  confers  with  Bart  Everett  regarding  countermine  strategies  that  could 
potentially  be  addressed  by  robotic  systems.  Code  2371  would  later  be  involved  in  a  number  of  mine 
detection,  marking,  excavation,  and  neutralization  efforts,  as  further  discussed  in  the  Project 
Summaries  sections  of  Volumes  2  and  3. 
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Following  the  1 1  September  2001  terrorist  attack  on  the  World  Trade  Center,  Code  2371  focused 
on  expediting  the  spiral  development  of  warfighter  needs  in  the  technical  community,  including 
academia,  industry,  and  DoD/govemment  entities.  By  2002,  the  emphasis  was  on  man-portable 
robots  used  in  counter-IED  missions,  resulting  in  the  branch’s  designation  as  a  “Center  of  Excellence 
for  Small  Robots”  in  the  OSD  Joint  Robotics  Program  Master  Plan  (JRP,  2002).  The  highlight  that 
year  was  a  visit  from  Secretary  of  Defense  Donald  Rumsfeld  on  26  August  (Figure  23),  prompted  by 
a  recommendation  from  Newt  Gingrich. 


Figure  23.  Robin  Laird  (foreground  left)  and  Bart  Everett  (foreground  right)  brief  Secretary  of 
Defense  Donald  Rumsfeld  on  Code  2371 ’s  support  for  Operation  Enduring  Freedom  during  his  visit 
to  the  Center  on  26  August  2002.  Rear  Admiral  Ken  Slaght,  Space  and  Naval  Warfare  Systems 
Command  (SPAWAR)  commander,  is  at  background  left. 

Some  17  robots  were  presented  from  eight  different  unmanned-ground- vehicle  projects,  with  an 
emphasis  on  various  systems  developed  for  force-protection  and  tactical  applications,  particularly 
those  used  in  Operation  Enduring  Freedom.  Also  demonstrated  was  the  Center’s  Urban  Robot 
(URBOT),  which  deployed  to  Afghanistan  with  Navy  Explosive  Ordnance  Disposal  Mobile  Unit  3  in 
April  2002  (see  later  Man-Portable  Robotic  System  section).  Assisting  was  Colonel  Bruce  Jette 
(Figure  24),  who  led  the  Robotic  Tiger  Team  that  performed  a  quick-response  technology  insertion  in 
Afghanistan  that  summer,  placing  small-robot  systems  in  the  hands  of  warfighters  engaged  in  life- 
threatening  roles. 
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Figure  24.  Colonel  Bruce  Jette  (right),  recently  returned  from  Afghanistan  and  future  head  of  the 
U.S.  Army  Rapid  Equipping  Forced  (REF),  assisted  in  the  presentation  to  Secretary  of  Defense 
Rumsfeld. 

On  15  May  2002,  Chief  of  Naval  Operations  Admiral  Vem  Clark  and  several  members  of  his  staff 
visited  and  were  briefed  on  a  number  of  key  Center  programs,  then  given  an  unmanned  systems 
demonstration  by  Robin  Laird,  Code  2371  Branch  Head  (Figure  25a).  The  following  week,  the 
Honorable  Hansford  T.  Johnson,  acting  secretary  of  the  Navy,  toured  the  Center,  receiving  an 
unmanned  systems  briefing  from  Bart  Everett,  Code  237’s  Associate  Division  Head  for  Robotics, 
followed  by  a  demonstration  of  man-portable  robots  from  the  Robotic  Systems  Pool  (Figure  25b). 
Considerable  interest  was  expressed  in  several  of  the  19  projects  then  underway,  with  focused 
attention  on  support  to  Navy  SEALS  and  Explosive  Ordnance  Disposal  (EOD)  units  deployed  in  Iraq 
and  Afghanistan  (Mullens,  2003b). 


Figure  25.  a)  Code  2371  Branch  Head  Robin  Laird  (right)  and  EMI  Thomas  Hoover  (left) 
demonstrate  the  URBOT for  CNO  Admiral  Vern  Clark  on  15  May  2003.  b)  Associate  Division  Head 
for  Robotics  Bart  Everett  (center)  and  Branch  Head  Robin  Laird  (right)  brief  the  Honorable  Hansford 
T.  Johnson,  acting  secretary  of  the  Navy,  the  following  week. 
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Rear  Admiral  Albert  “Bert”  Calland,  Commander,  Naval  Special  Warfare  Command  (NSWC) 
briefed  senior  management  and  other  interested  parties  on  Special  Operations  during  Operation 
Enduring  Freedom  in  Afghanistan  on  28  May  2003  (Figure  29).  Admiral  Calland  and  his  staff  were 
presented  various  briefings  by  Center  personnel,  with  a  focus  on  intelligence,  surveillance  and 
reconnaissance  (ISR)  information  supporting  special  operations.  An  important  part  of  the  latter 
included  a  visit  to  the  Robotic  Systems  Branch  for  live  demonstrations  of  unmanned  systems  for 
surface,  ground,  air,  and  underwater  vehicle  applications  (Mullens,  2003b). 


Figure  26.  SPAWAR  Commander  Rear  Admiral  Ken  Slaght  (center),  and  Executive  Director  Dr.  Bob 
Kolb  (right),  host  Rear  Admiral  Bert  Calland,  Naval  Special  Warfare  Command  (left),  during  his  visit 
to  the  Center  on  28  May  2003. 

Beginning  in  2004,  the  Robotic  Systems  Branch  deployed  several  active-duty  officers  and 
Reservists  to  Iraq  and  Afghanistan  to  support  UGV  operations  by  Joint  Forces  EOD  units  (Figure 
27),  to  include  establishment  of  the  Joint  Robotics  Repair  Facility  for  the  RS  JPO  at  Camp  Victory, 
Iraq  (see  also  Navy  Unmanned  Systems  Reserve  Unit  section,  Volume  2).  This  assignment  turned  out 
to  be  a  wonderful  opportunity  to  gather  valuable  user  feedback,  which  is  often  hard  to  obtain.  That 
same  year,  the  Robotic  Systems  Branch  (Code  2371)  was  renamed  the  Unmanned  Systems  Branch, 
which  was  later  redesignated  as  Code  7171  in  2007  (Appendix  E). 


23 


Figure  27.  IT2  Jennifer  Smith  and  ATI  Jim  Overton  of  the  SPAWAR  Unmanned  Systems  Reserve 
Unit  are  recognized  by  the  184th  Ordnance  Battalion  (EOD)  for  their  services  at  Camp  Victory 
(October  2004  through  April  2005). 

On  24  February  2005,  Brigadier  General  Stephen  Reeves  (U.S.  Army),  Joint  Program  Executive 
Officer  for  Chemical  and  Biological  Defense  (PEOCBD),  visited  the  Unmanned  Systems  Branch  to 
become  familiar  with  Code  2371 ’s  technical  capabilities  (Figure  28).  He  was  accompanied  by 
Colonel  Camille  Nichols,  program  manager  for  Guardian  (PM  Guardian).  She  had  visited  the  our  lab 
the  previous  fall  (Thomas,  2004).  Brigadier  General  Reeves  and  Colonel  Nichols  were  given  status 
updates  on  two  key  Army  programs  the  Center  was  supporting  for  the  Product  Manager,  Force 
Protection  Systems  (PM-FPS):  the  Mobile  Detection  Assessment  Response  System  and  the  Family  of 
Integrated  Rapid  Response  Equipment  (Mullens,  2005). 


Figure  28.  Bart  Everett  (right)  describes  the  operation  of  the  marsupial  payload  mounted  on  the  back 
of  an  MDARS  Exterior  robot  (BAA  version)  to  Brigadier  General  Reeves  (center)  as  a  man-portable 
URBOT  is  recovered  for  transport.  Navigation  and  Applied  Sciences  Department  Head  Dr.  Frank 
Gordon  is  at  background  right. 
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On  1 1  May  2005,  SPAWAR  Commander  Rear  Admiral  Kenneth  Slaght  hosted  a  visit  to  the 
Unmanned  Systems  Group  for  SPECWARCOM  Deputy  Commander  Captain  John  McTighe, 
and  his  science  advisor,  John  Young  (Mullens,  2005).  The  purpose  of  this  visit  was  to  view  the 
innovative  stabilization  technology  of  Motion  Picture  Marine  of  Marina  del  Rey,  CA.  In  2003, 
SSC  San  Diego  acquired  one  of  the  company’s  Perfect  Horizon  stabilization  systems  for 
incorporation  on  our  Unmanned  Surface  Vehicle  (USV)  project,  which  has  since  been  in  use 
stabilizing  camera/sensor  systems  aboard  the  USV  in  both  pitch  and  roll.  The  Navy  SEALs  were 
interested  in  stabilization  of  camera  sensor  systems  and  potentially  also  weapons  (Figure  29). 


Figure  29.  Mike  Bruch  (foreground  right)  explains  the  Perfect  Horizon  active-stabilization  mount  for  a 
0.50-caliber  M2  machine  gun  to  SPAWAR  Commander  Rear  Admiral  Ken  Slaght  (background  right), 
and  Captain  John  McTighe,  deputy  commander  of  Naval  Special  Warfare  Command  (background 
center).  Captain  John  Barron,  executive  officer  of  Space  and  Naval  Warfare  Systems  Center  San 
Diego,  is  at  background  left. 

On  19  November  2008,  the  Unmanned  Systems  Branch  hosted  a  visit  from  Major  General  Robert 
G.F.  Lee,  adjutant  general  of  the  Department  of  Defense  for  the  state  of  Hawaii,  and  Clifton  Cheng 
of  the  Naval  Undersea  Warfare  Center  Detachment  (Figure  30a).  Following  a  high-level  overview  of 
the  branch’s  history  and  mission  by  Bart  Everett,  Major  General  Lee  received  an  in-depth 
presentation  from  Robin  Laird  on  the  Joint  Battlespace  Command  and  Control  System,  recently 
demonstrated  at  the  Force  Protection  Joint  Experiment.  He  was  also  briefed  on  the  Robotics  Systems 
Pool  by  John  Andrews  (Figure  30b),  and  the  Networked  Remotely  Operated  Weapon  System  project 
by  electrical  engineer  Amin  Rahimi. 
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Figure  30.  a)  Robin  Laird  (background  center)  briefs  Major  General  Robert  Lee  (foreground  right),  on 
the  Joint  Battlespace  Command  and  Control  System  used  at  the  Force  Protection  Joint  Experiment. 
b)  John  Andrews  answers  questions  following  his  comprehensive  overview  of  the  Robotic  Systems 
Pool  managed  by  the  Unmanned  System  Reserve  Unit. 

On  October  27th,  2009,  SPAWAR  Commander  Rear  Admiral  Michael  Bachmann  hosted  a  VIP 
visit  to  the  Unmanned  Systems  Branch  for  J.M.  “Raleigh”  Durham  of  the  Pentagon’s  Director  of 
Defense  Research  and  Engineering  (DDR&E).  Following  a  high-level  introductory  overview  by  Bart 
Everett,  selected  branch  projects  related  to  vehicle  autonomy  and  collaboration  were  presented  by 
Branch  Head  Hoa  Nguyen  (Curd,  2009).  The  principal  focus  of  Mr.  Durham’s  visit,  however,  was  on 
live  demonstrations  of  autonomous  unmanned  vehicles  across  all  operational  domains  of  air,  land, 
and  sea,  with  command  and  control  provided  by  the  Multi-robot  Operator  Control  Unit  (MOCU) 
software  (Figure  31). 


Figure  31.  Estrellina  Pads  (foreground  right)  briefs  Rear  Admiral  Michael  Bachman  (center)  and 
J.M.  “Raleigh”  Durham  (background  left)  on  the  Urban  Environment  Exploration  (URBEE)  project 
in  Building  624  on  27  October  2009,  with  URBEE  software  engineer  Donnie  Fellers  overseeing 
the  demonstration  (foreground  left). 
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On  10  June  2010,  the  Center  hosted  prospective  SPAWAR  Commander  Rear  Admiral  Patrick  H. 
Brady,  who  took  command  of  the  Space  and  Naval  Warfare  Systems  Command  in  August  that  same 
year.  One  of  the  stops  on  his  agenda  included  the  Unmanned  Systems  Group,  where  he  was  briefed 
by  Bart  Everett  on  how  unmanned  systems  fit  the  Center’s  role  in  command,  control,  and 
communications.  The  walk-through  systems  tour  illustrated  how  some  degree  of  vehicle  autonomy 
facilitated  command  and  control  while  reducing  the  communication  bandwidth,  and  how  the  on¬ 
board  sensors  supporting  such  autonomy  could  significantly  contribute  to  information  warfare 
(Figure  32). 


Figure  32.  Bart  Everett  briefs  prospective  SPAWAR  Commander  Rear  Admiral  Patrick  Brady  on  the 
commercial  3D  stereo-camera  pair  and  Velodyne  64  lidar  employed  on  the  six-wheel  MAXATV 
(left),  in  contrast  with  the  massive  custom-built  counterpart  used  on  the  earlier  MDARS-Exterior 
vehicle  in  background. 

As  more  and  more  employees  were  brought  on  board  to  meet  the  expanding  project  load,  a  split 
into  two  branches  was  required  in  2012  under  the  Code  717  Advanced  Systems  and  Applied  Sciences 
Division:  Code  71710  under  Branch  Head  Tracy  Heath,  and  Code  71720  under  Branch  Head  Rachel 
TenWolde.  These  two  branches,  which  collectively  came  to  be  known  as  the  Code  717  Unmanned 
Systems  Group,  remained  high  on  the  list  of  potential  tour  opportunities  for  Center  visitors  and  VIPs. 
The  most  recent  highlight  in  this  regard  was  a  visit  from  Secretary  of  Defense  Ashton  B.  Carter  on  3 
February  2016  (Figure  33). 
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Figure  33.  a)  Code  71710  Branch  Head  Tracy  Heath-Pastore  greets  Secretary  of  Defense  Ashton 
Carter  during  his  visit  on  3  February  2016.  b)  Secretary  of  Defense  Carter  coins  a  Reconnaissance 
and  Detection  Expendable  Rover  (RaDER)  vehicle  designed  by  SSC  Pacific  engineers  to  support 
U.S.  Marine  Corps  battlefield  operations. 


Carter  was  visiting  the  Center  to  meet  with  SSC  Pacific’s  workforce,  tour  the  Center’s  Battlespace 
Exploitation  Mixed  Reality  (BEMR)  lab,  and  gain  a  deeper  understanding  of  the  many  futuristic 
technologies  and  systems  being  developed  to  ensure  America’s  warfighters  have  a  strategic  and 
technological  advantage.  Following  the  visit,  Tracy  Heath-Pastore  and  Mike  Bruch  further  briefed 
SPAWAR  Commander  Rear  Admiral  David  H.  Lewis  on  ONR-funded  unmanned  ground  vehicle 
work  (Figure  34),  which  represents  state  of  the  art  in  off-road  and  expeditionary  autonomous 
navigation.  (See  further  discussion  of  these  efforts  in  the  Project  Summaries  section  of  Volume  3.) 


Figure  34.  Code  717  Chief  Engineer  for  Robotics  Mike  Bruch  (center)  holds  the  commemorative  coin 
presented  to  the  RaDER  vehicle  by  SECDEF  while  describing  the  ONR-funded  project  to  Rear 
Admiral  David  H.  Lewis,  the  new  SPAWAR  commander,  on  3  February  2016. 
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TECHNOLOGY  FOCUS  AREAS 


The  Department  of  the  Navy  Objectives  for  FY  2012  and  Beyond  contained  the  following 
paragraph  (DoN,  2012): 

“5.  Dominate  in  Unmanned  Systems 

a.  Integrate  Unmanned  Systems  into  the  DON  Culture 

b.  Develop  Unmanned  Systems  in  the  Air 

c.  Deploy  and  Establish  Unmanned  Systems  on/under  Sea 

d.  Field  Unmanned  Systems  on  the  Ground” 

Three  years  later,  the  Department  of  the  Navy  Objectives  for  FY-15  and  Beyond  reiterated  this 
continuing  objective  as  follows  (DON,  2015): 

“5.  Proliferate  Unmanned  Systems 

a.  Integrate  Unmanned  Systems  into  the  DON  Culture 

b.  Develop,  Field,  and  Deploy  Unmanned  Systems  in  the  Air,  on/under  the  Sea,  and  on  the 
Ground” 

By  this  time,  unmanned  systems  had  increasingly  been  employed  in  various  operations  ranging 
from  combat  to  disaster-relief  across  all  operational  domains  of  air,  land,  and  sea,  with  growing 
appreciation  of  the  results. 

This  wide  variety  of  applications  required  configurable  command,  control,  and  communication 
(C3)  systems  designed  to  fit  the  assigned  mission,  capable  of  clearly  displaying  relevant  mission- 
specific  information  to  the  warfighter.  Across  this  spectrum  of  opportunity,  we  identified  five 
common  technology-focus  areas  that  over  time  have  evolved  into  core  business  units  (Figure  35), 
greatly  facilitating  inter-project  technology  transfer  and  subsequent  system  interoperability.  The 
degree  of  information  dominance  ultimately  achieved  by  unmanned  systems  is  a  direct  function  of 
how  these  factors  interrelate  and  are  consequently  exploited,  as  will  be  further  discussed  in  the 
following  subsections. 
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Figure  35.  The  degree  of  information  dominance  achieved  by  unmanned  systems  is  directly 
influenced  by  many  interrelated  factors  that  must  be  well  understood  and  properly  integrated. 

The  Center’s  mantra  is  command,  control,  and  communications,  and  providing  such  for  unmanned 
vehicles  can  be  greatly  facilitated  if  these  systems  have  some  degree  of  supervised  autonomy,  as 
opposed  to  just  teleoperated.  Enabling  intelligent  on-vehicle  autonomy  requires  appropriate  local 
perception,  typically  from  radar,  lidar,  sonar,  and/or  vision-based  sensors.  A  “smart”  unmanned 
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system  must  effectively  perceive  its  surroundings  to  make  appropriate  behavioral  decisions,  and  in 
the  early  history  of  UGV  development,  this  was  the  dominating  theme.  More  recently,  however, 
considerable  attention  has  been  given  to  exploiting  on-board  perception  capability  for  purposes  of 
intelligence,  surveillance,  and  reconnaissance  (ISR),  through  data  mining  and  visualization.  Each  of 
these  five  business  units  will  be  further  discussed  in  the  following  subsections. 

Command,  Control,  and  Communications 

SSC  Pacific’s  Unmanned  Systems  Group  has  developed  command,  control,  and  communication 
(C3)  systems  for  unmanned  vehicles  and  sensors  for  over  35  years,  and  is  currently  supporting 
acquisition  programs  in  several  services.  The  Multiple  Resource  Host  Architecture  (MRHA),  for 
example,  provided  C3  for  the  US  Army  Product  Manager  -  Force  Protection  Systems  (PdM-FPS)  on 
the  Mobile  Detection  Assessment  Response  System  (MDARS)  program  of  record.  MDARS  envisioned 
up  to  32  interior  and  exterior  security  robots  patrolling  INCONUS  Government  sites,  with  a  later 
force-protection  variant  of  the  concept  evaluated  for  possible  USMC  deployment. 

The  Multi-robot  Operator  Control  Unit  (MOCU)  evolved  as  a  tactical  derivative  of  the  MRHA 
(Figure  36).  Originally  developed  to  support  the  Man  Portable  Robot  System  (MPRS)  program  for 
PMS-EOD,  MOCU  was  later  selected  to  provide  C3  for  the  prototype  USV  mission  modules  on  the 
Navy’s  Littoral  Combat  Ship  (LCS).  The  modularity  of  MOCU  facilitated  extension  to  meet  the 
individual  goals  of  multiple  projects,  with  the  inherent  ability  to  quickly  adapt  to  emergent  needs. 
With  over  70  MOCU  modules  developed  by  multiple  partners,  the  software  is  able  to  control  ground, 
surface,  air,  and  underwater  vehicles.  The  extremely  flexible  user  interface  features  a  game-like  look 
and  feel,  with  multiple  users  spanning  the  full  range  from  academia  to  formal  DoD  programs  of 
record. 


Figure  36.  Cathy  Mullens  (left)  and  Bart  Everett  (right)  brief  Dr.  Dave  Thomas  of  TARDEC  on  the 
Center’s  evolution  of  unmanned-vehicle  command  and  control.  The  wall  poster  shows  a  timeline  for 
MRHA  expansion  into  MOCU,  which  by  2005  was  already  being  used  by  a  number  of  different 
organizations  within  industry  and  DoD. 

Effective  communication  is  obviously  necessary  to  link  the  command-and-control  architecture 
with  the  relevant  assets,  which  could  well  include  a  mix  of  unmanned  vehicles,  conventional  manned 
systems,  and  distributed  sensors.  During  Operations  Iraqi  Freedom  and  Enduring  Freedom,  the 
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emphasis  was  on  addressing  the  C3  needs  of  man-portable  robots  (Figure  37),  which  were  deployed 
in  large  numbers  to  address  the  growing  IED  threat.  The  high-bandwidth  RF  communications  links 
for  these  systems  required  line  of  sight,  which  presented  numerous  problems  for  teleoperation, 
especially  for  the  small  man-portable  class  of  UGVs  (Nguyen  et  al.,  2013).  Their  inherent  low 
antenna  height  causes  signal  occlusion  due  to  dips  and  rises  in  the  terrain,  which  can  also 
significantly  reduce  the  Fresnel-zone  clearance,  and  operation  in  urban  environments  typical  of  Iraq 
only  exacerbates  the  problem  (Figure  37). 


Figure  37.  Joint  Urgent  Operational  Needs  Statement  (JUONS)  CC-0333  called  for  reliable 
communications  between  a  Man  Transportable  Robotic  System  (MTRS)  operating  inside  a  building 
and  its  operator  stationed  outside.  This  critical  need  was  addressed  by  the  Automatically  Deployed 
Communication  Relays  (ADCR)  project  in  2010. 

In  the  early  2000s,  the  DARPA  Mobile  Autonomous  Robot  Software  (MARS)  program  funded  the 
Center  to  develop  ad  hoc  networking  radios  and  software  under  the  Autonomous  Mobile 
Communication  Relays  (AMCR)  project.  This  technology  was  later  leveraged  on  a  more  practical  and 
logistically  simpler  system,  the  Automatically  Deployed  Communication  Relays  (ADCR)  project. 
Funded  by  the  Joint  Ground  Robotics  Enterprise,  several  generations  of  ADCR  systems  introduced 
increasingly  more  capable  hardware  and  software  for  automatic  communication-link  maintenance  by 
deploying  static  relay  nodes  from  mobile  robots.  This  capability  was  tapped  in  2010  to  fulfill  an 
urgent  need  from  theater,  resulting  in  243  kits  of  ruggedized  robot-deployable  communication  relays 
that  were  sent  to  Afghanistan  to  extend  the  range  of  EOD  and  tactical  ground  robots  in  2012  (Nguyen 
et  al.,  2013). 

Autonomy 

The  Code  717  Unmanned  Systems  Group  has  developed  autonomous  vehicles  (air,  land,  and  sea) 
for  over  30  years,  more  recently  specializing  in  mapping,  localization,  path  planning,  and  route 
execution  in  complex  GPS-denied  indoor  and  outdoor  environments.  The  branch’s  first  autonomous 
UGV  was  the  Ground  Surveillance  Robot,  which  in  1986  could  automatically  follow  a  pedestrian  or 
slowly  moving  lead  vehicle  with  no  human  intervention  (Gage,  1995b).  In  the  pre-GPS  era  of 
unmanned  systems  development,  however,  this  early  research  surrogate  also  lacked  dead-reckoning 


31 


encoder  input  and  carried  no  inertial  navigation  unit,  and  thus  was  incapable  of  accurate 
localization.12 


Figure  38.  The  Ground  Surveillance  Robot  (GSR)  was  a  surplus  M-114  Command  and 
Reconnaissance  Carrier  outfitted  by  Code  442  to  serve  as  a  surrogate  testbed  for  autonomous 
navigation  and  collision  avoidance,  circa  1986. 

For  indoor  applications,  more  sophisticated  autonomous  navigation  strategies  were  introduced  at 
the  Center  with  the  arrival  of  ROBART I  (Figure  39a)  and  ROBART II  (Figure  39b)  in  late  1986. 13 
The  mapping,  localization,  path  planning,  and  collision-avoidance  algorithms  developed  by  Gary 
Gilbreath  on  ROBART II  were  subsequently  improved  for  incorporation  on  the  Army’s  Mobile 
Detection  Assessment  and  Response  Interior  ( MDARS-I)  robotic  security  program  in  1990  (Figure 
39c).  The  MDARS-Interior  robot  autonomy  was  ported  to  the  MDARS-Exterior  robot  in  1995,  and 
transitioned  in  2003  to  our  Unmanned  Surface  Vehicle  (USV)  project  for  supervised  autonomous 
operation  in  the  maritime  domain. 


12  It  was,  however,  equipped  with  an  agricultural  John  Deere  Doppler  radar  for  measuring  forward  velocity,  and  a 
modified  laser  rangefinder  intended  for  landmark  referencing. 

13  ROBART  I  demonstrated  fully  autonomous  indoor  navigation  and  room-level  localization  in  1981  (Everett,  1982; 
2005),  followed  by  a  much  improved  ROBART II  in  1987  (Everett  et  al.,  1988;  Everett  &  Gilbreath,  1989). 
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a) 


b) 


c) 


Figure  39.  a)  ROBART I  (1980-1982).  b)  ROBART II  (1982-1992).  c)  MDARS- Interior  prototype 
(circa  1997). 


The  term  “autonomy”  is  often  overused  and  less  often  well  understood,  however,  and  the  popular 
descriptor  “fully  autonomous”  even  more  so.  While  a  detailed  discussion  of  this  particular  issue  is 
given  under  the  Technology  Transfer  project  (2003-2007)  in  the  chronological  Project  Summaries 
section  of  Volume  2,  for  introductory  purposes  here  we  shall  focus  on  autonomy  as  applied  to 
unmanned  vehicle  navigation.  To  further  simplify  things,  the  discussion  will  be  constrained  to 
unmanned  ground  vehicles  (UGVs),  with  implied  extension  to  other  operational  domains  such  as  air 
and  sea. 

That  being  said,  autonomous  navigation  for  UGVs  can  be  generally  decomposed  into  four  key 
components:  1)  mapping,  2)  localization,  3)  path  planning,  and  4)  route  execution.  Mapping  involves 
generating  some  type  of  world  model  or  map  representation  in  which  the  unmanned  vehicle  can  be 
localized,  and  which  supports  subsequent  path  planning  to  generate  a  series  of  waypoints  leading  to 
some  desired  destination.  While  early  UGVs  relied  upon  a  priori  map  data,  real-world  operations 
require  simultaneous  localization  and  mapping  (SLAM),  in  which  the  moving  vehicle  accurately 
perceives  its  surroundings  to  build  its  own  map  (Figure  40). 
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Figure  40.  This  robot-generated  SLAM  map  (left)  of  Battery  Woodward  (right),  a  15-room 
underground  WWII  bunker  located  on  the  Unmanned  Systems  Group’s  Central  Test  Site,  was 
created  from  lidar  range  data  collected  by  a  Sick  L/WS-200-equipped  iRobot  ATRV as  it 
autonomously  explored  the  previously  unmapped  structure,  circa  May  2004. 

The  localization  component  involves  incremental  determination  of  vehicle  position  and  orientation 
along  the  route  being  executed,  and  there  are  a  number  of  synergistic  strategies  that  make  this 
possible,  to  include  SLAM  (Figure  40)  and  inertial  measurement  (Figure  41a),  with  a  Kalman  filter 
or  fuzzy  logic  (Figure  41b)  to  optimally  fuse  the  results.  Path  planning  algorithms  work  on  the  world 
model  representation  (map)  to  calculate  the  most  effective  or  desired  route  to  achieve  the  destination 
goal,  to  include  dynamic  route  replanning  in  response  to  actual  real-world  conditions  encountered 
during  transit.  Once  the  path  planner  has  determined  an  appropriate  set  of  waypoints  defining  the 
desired  path  segment,  the  route-execution  component  encompasses  all  the  behaviors  required  to 
successfully  traverse  from  point  A  to  point  B  to  point  C,  etc. 


Figure  41 .  a)  The  University  of  Michigan’s  FLEXnav  PPE  (proprioceptive  position  estimation) 
precision  dead  reckoning  system  is  shown  installed  on  an  ActivMedia  P2AT  (Ojedo,  Reina,  Cruz, 
&  Borenstein,  2006).  b)  Comparison  of  Kalman-filter  (pink)  and  fuzzy-logic  (blue)  fusion  following 
a  round-robin  run  that  started  at  the  graph  origin  (Pads  et  al.,  2006). 
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Probably  the  most  obvious  of  these  behaviors  is  collision  avoidance,  the  intent  being  to  safely 
execute  each  route  segment  without  running  into  anything.  Positive  (above-ground)  static  obstacles 
are  the  easiest  to  detect,  typically  using  lidar,  radar,  and/or  video-based  sensors,  with  predictive 
avoidance  of  moving  objects  a  bit  more  difficult.  Detection  of  negative-obstacles  (below-grade 
hazards  such  as  ditches  or  potholes)  is  tougher  still,  as  they  are  largely  occluded  from  view  and  hence 
hard  to  spot  at  a  distance,  particularly  from  fast-moving  or  low-profile  vehicles  (Figure  42).  Effective 
slope  determination  is  required  for  establishing  ground  truth  in  both  the  above  scenarios,  as  well  as 
for  warning  of  unsafe  terrain  conditions. 


Figure  42.  The  red-colored  areas  in  the  inset  at  upper  left  correspond  to  perceived  negative 
obstacles  in  the  robot’s  path,  based  on  3D  lidar  scans  of  the  unpaved  road  as  shown  in  the  main 
image. 

Our  more  recent  development  of  tip-over  detection,  warning,  and  prevention  behaviors  that 
analyze  inertial  forces  acting  upon  the  vehicle  further  lessen  the  chances  of  mishap  (Figure  43).  The 
software’s  ability  to  detect  the  vehicle  model  and  any  installed  payloads  can  enable  dynamic 
recalculation  of  the  center  of  gravity,  even  while  in  motion,  so  corrective  action  can  be  taken  to  avoid 
accidents.  This  algorithm  can  be  extended  to  accommodate  proactive  center-of-gravity 
reconfiguration  to  push  the  envelope  of  traversibility,  as  for  example  extending  the  manipulator 
forward  when  climbing  stairs. 


Figure  43.  Developed  by  Aaron  Burmeister,  Leah  Kelly,  and  Kurt  Talke,  the  tip-over  prevention 
algorithm  assesses  the  various  accelerations  acting  upon  the  robot  relative  to  its  center  of  gravity 
(lower  left  quadrant)  as  it  traverses  the  slope  shown  at  upper  right  to  detect,  warn  of,  and/or  prevent 
an  impending  accident. 


35 


Small-robot  autonomy  involving  the  above  navigational  components  has  always  been  a  challenge, 
for  three  reasons:  1)  limited  space  and  payload  capacity,  2)  limited  available  energy,  and  3)  close 
proximity  to  the  ground,  which  complicates  both  perception  and  communications.  Addressing  these 
issues  became  the  highest  priority,  as  man-portable  EOD  robots  were  being  deployed  in  large 
numbers  to  Afghanistan  and  Iraq  to  address  the  growing  IED  threat.  Accordingly,  the  group  focused 
on  expediting  the  development  of  autonomous  navigation  for  man-portable  robots  (Figure  44),  as 
well  as  the  small  lightweight  perception  sensors  required  to  support  same  (Figure  45). 


Evolution  of  Man-Portable  Robot  Autonomy 

A  CCAT  Success  Story 


Figure  44.  The  development  of  man-portable  robot  autonomy  for  EOD  applications  was  expedited  by 
a  focused  effort  that  grew  a  2004  laboratory  prototype  into  a  hardened  system  on  a  commercial 
robot  by  2009,  which  was  competitively  selected  in  response  to  a  Joint  Urgent  Operational  Needs 
Statement  (JUONS)  the  following  year. 

Perception 

Localization,  path  planning,  and  route-execution  behaviors  such  as  collision  avoidance  and  tip- 
over  prevention  rely  heavily  on  perception  to  collect  the  geometric  data  required  for  creating  high- 
fidelity  world  models  that  support  autonomous  navigation.  The  Unmanned  Systems  Group’s  use  of 
the  CommonSense  cross-platform  library  facilitates  integration  of  3D  sensors  with  the  modular 
Autonomous  Capabilities  Suite  (ACS)  and  Robot  Operating  System  (ROS)  unmanned-vehicle 
architectures.  Primary  focus  areas  include  2D  and  3D  lidar,  monocular/stereo  vision,  and  radar  sensor 
systems  collectively  supporting  the  following: 

•  Multi-view  stereo 

•  360-degree  camera  arrays 

•  Fused  3D  lidar  and  color  images 

•  Point-cloud  processing 

•  Spatial-phase  sensing 

•  Large-scene  and  small-scene  2D  and  3D  modeling 

•  Feature  estimation,  surface  reconstruction,  and  segmentation 
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Typical  applications  include: 

•  Enhanced  depth  perception  and  improved  operator  situational  awareness 

•  3D  visualization  for  man-portable  EOD  robots 

•  Positive  and  negative  obstacle  detection  and  avoidance 

•  Visual  odometry  for  enhanced  dead  reckoning 

•  GPS-denied  navigation  and  localization 

•  Terrain-traversibility  assessment 

•  Autonomous  stair  detection,  climbing,  and  descending 

•  Large-scale  multi-story  mapping 

•  Human  motion  and  presence  detection 

•  Tunnel  exploration,  mapping,  and  characterization 

•  Terrestrial  and  maritime  force  protection  and  security 

•  Intelligence,  surveillance,  and  reconnaissance  (ISR) 

The  evolution  of  perception  sensors  supporting  the  man-portable  robot  autonomy  advancements 
shown  earlier  in  Figure  44  is  similarly  depicted  in  Figure  45. 


Figure  45.  The  development  of  practical  perception  sensors  to  support  man-portable  robot  autonomy 
was  challenged  by  the  inherent  size,  weight,  and  power  restrictions  of  small  UGV  platforms. 

Data  Mining 

Prior  to  the  drawdown  of  coalition  forces  in  Iraq  and  Afghanistan,  almost  8,000  counter-IED 
UGVs  were  operating  on  the  ground  in  theater,  each  sending  back  15  frames  of  video  per  second. 
This  video  data  for  the  most  part  was  not  retained  after  the  mission,  as  the  primary  role  of  the  real¬ 
time  perception  was  to  provide  operator  situational  awareness  during  teleoperation.  The  later 
introduction  of  limited  autonomous  navigation  allowed  lidar  and  stereo-based  perceptual  support  for 
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certain  intelligent  behaviors  that  reduced  the  driving  burden  of  the  human  operator,  but  to  our 
knowledge  the  video  stream  was  never  used  for  such  purposes  in  theater. 

Thousands  of  UAVs  were  also  flying  in  the  skies  above  American  troops  on  the  ground,  collecting 
and  sending  back  even  higher  resolution  video  imagery.  Most  of  this  data  was  saved,  as  the  ISR 
mission  was  the  main  reason  the  majority  of  these  airborne  drones  had  been  launched  in  the  first 
place.  Analysts  pouring  over  this  flood  of  incoming  imagery  were  soon  overwhelmed,  however, 
resulting  in  huge  backlogs  that  forced  the  prioritization  of  post-mission  analysis,  which  in  turn 
reduced  effectiveness  and  caused  some  degree  of  potentially  relevant  intel  to  inadvertently  fall 
through  the  crack. 

In  retrospect,  the  collective  employment  of  large  numbers  of  unmanned  air  and  ground  systems 
can  obviously  generate  an  enormous  amount  of  valuable  ISR  data,  which  could  be  “mined”  post¬ 
mission  to  great  tactical  advantage.  There  currently  are  three  fundamental  problems  hindering 
practical  attainment  of  this  goal,  however,  the  first  being  it  cannot  be  done  quickly  through 
conventional  analysis.  Secondly,  effectively  reviewing  archived  video  data  for  targeted  mission¬ 
relevant  intelligence  on  short  notice  is  also  impractical  using  current  practice.  And  thirdly,  even  if 
such  timely  and  targeted  retrieval  were  possible,  traditional  storage  requirements  for  so  much  video 
data  would  quickly  become  unmanageable. 

Our  growing  experience  with  3D  world  modeling  in  support  of  autonomous  UGV  navigation 
suggested  a  very  promising  potential  solution  to  all  three  of  the  above  problems.  In  2012,  the  3D 
Visualization  project  developed  an  ability  to  stitch  successive  UGV  video  frames  together  in  near- 
real-time  to  create  an  accurate  three-dimensional  representation  of  a  small  volume  of  interest,  such  as 
a  suspected  IED  site  (Figure  46).  This  achievement  was  accomplished  using  structure-from-motion 
and  feature-tracking  algorithms  developed  by  the  University  of  North  Carolina  (UNC). 


Figure  46.  Created  by  stitching  successive  video  frames  together  in  2  minutes  or  less,  this  3D  model 
of  a  training  IED  could  be  viewed  by  the  UGV  operator  from  any  desired  perspective  for  detailed 
near-real-time  analysis. 

A  post-processing  approach  developed  by  the  University  of  Washington  for  the  Unmanned 
Systems  Group’s  Urban  Environment  Modeling  project  leveraged  the  UNC  reconstruction 
algorithms  and  high-powered  graphics-processing-unit  (GPU)  pipeline  for  stitching  together 
asynchronous  still  images  and  video  frames.  The  resulting  capability  to  model  a  building  (Figure  47) 
was  later  expanded  to  cover  a  full  city  block.  This  early  work  was  based  on  the  Photo  Tourism 
research  project  pursued  by  graduate  student  Noah  Snavely  at  the  University  of  Washington 
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(Wikipedia,  20 15), 14  which  was  brought  under  contract  for  this  project  through  the  JGRE  Robotics 
Technology  Consortium. 


Figure  47.  Numerous  asynchronously  gathered  35-mm  still  shots,  such  as  this  upper  image  of  a 
“battle-damaged”  structure  at  the  Camp  Pendleton  MOUT  site,  were  post-processed  to  generate  the 
3D  representation  at  bottom.  The  user  could  intuitively  fly  through  this  model  using  the  mouse  to 
more  closely  view  regions  of  specific  interest. 

In  2006,  the  Photo  Tourism  project  used  a  huge  assortment  of  asynchronously  gathered  Web 
images  of  varying  resolution  and  standoff  to  generate  a  3D  model  and  point  cloud  of  famous  world 
landmarks,  as  for  example  the  Eiffel  Tower  and  Notre  Dame  Cathedral.  Such  a  process  could  be 
continuously  applied  to  the  offline  aggregation  of  individual  video  frames  collected  from  air  and  land 
vehicles,  creating  a  composite  3D  representation  that  provided  rapid  and  intuitive  access  to  a 
warfighter’s  specific  region  of  interest.  An  additional  benefit  would  be  an  upper  bound  on  storage 
requirements,  as  redundant  data  would  simply  be  discarded. 15 

Visualization 

As  previously  discussed,  effective  visualization  of  video  imagery  can  play  an  important  role  in  two 
primary  scenarios:  1)  improved  depth  perception  and  situational  awareness  for  the  immediate  user  in 
the  field,  and  2)  improved  situational  awareness  during  post-mission  analysis. 16  In  either  case,  the 
collected  perception  data,  while  perhaps  locally  valuable  to  the  unmanned  system  that  gathered  it, 
must  be  presented  to  the  user  in  a  manner  that  quickly  conveys  the  needed  information  in  intuitive 
fashion.  If  the  human  operator  cannot  readily  visualize  the  significance  of  what  he  or  she  is  looking 
at  on  the  display,  it  is  consequently  of  little  tactical  value  in  either  of  the  above  two  scenarios. 


14  Microsoft’s  desktop  application  Photosynth  is  a  refined  version  of  Photo  Tourism  (Wikipedia,  2015). 

15  While  this  would  be  a  data-heavy  and  time-consuming  process,  it  likely  will  become  more  practical  as  more 
capable  APUs  become  readily  available  and  new  parallel  processing  techniques  emerge. 

16  Other  potential  applications  include  training  and  forensics  (change  detection). 
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In  the  first  scenario,  appropriate  rendering  of  relevant  data  describing  the  task  at  hand  via  an 
optimal  man-machine  interface  can  greatly  enhance  operator  situational  awareness  in  ways  never 
before  possible,  as  illustrated  in  Figure  48.  Before  an  operator  cuts  a  command  wire  just  uncovered 
on  top  of  the  pallet,  for  example,  he  or  she  may  want  to  verify  what  color  wire  goes  where  on  that 
part  of  the  device  behind  the  pallet.  In  the  past,  this  meant  driving  the  robot  around  to  the  back  to 
physically  take  one  more  look,  as  opposed  to  manipulating  the  virtual  camera  perspective  to 
accomplish  the  same  goal  while  the  robot  stays  put. 


Figure  48.  This  3D  model  of  a  mock-up  suspected  IED  (left),  created  from  video  streams  collected 
by  the  stereo-camera  pair  (top),  17  has  been  merged  with  an  avatar  of  the  iRobot  PackBot,  which 
moves  in  concert  with  the  actual  robot  to  provide  the  operator  with  otherwise  unobtainable  near-real¬ 
time  situational  awareness. 

As  for  the  second  scenario,  the  ability  to  generate  large-scale  high-resolution  3D  models  of  the 
operating  environment  in  near  real  time  also  opens  up  new  opportunities  for  post-mission  ISR 
analysis,  which  are  significantly  enhanced  by  appropriate  rendering  of  the  data-mining  results.  Such 
models  could  also  support  automatic  change-detection  algorithms  that  would  substantially  reduce  the 
analysts’  workload  by  alerting  unusual  activity  that  could  then  be  visualized  on  a  prioritized  basis.  In 
addition  to  fulfilling  the  navigational  needs  of  supervised  autonomous  unmanned  air  and  ground 
vehicles,  3D  models  of  large  urban  environments  could  also  be  used  to  support  warfighter  training 
via  mission  simulation. 

KEY  SERVICE  AREAS 

The  U.S.  Department  of  Defense  spends  an  enormous  amount  of  money  each  year  on  the 
development  of  new  and  improved  technology,  and  an  even  larger  sum  of  money  is  annually 
expended  to  support  the  use  of  technology,  both  on  and  off  the  battlefield.  For  a  variety  of  reasons 
beyond  the  scope  of  this  document,  however,  the  most  persistent  problem  is  rapid  and  effective 
transition  of  newly  developed  technology  into  the  actual  hands  of  the  warfighter  in  theater.  In 
recognition  of  this  potentially  crippling  limitation  (commonly  known  as  the  “valley  of  death”),  we 
implemented  several  key  service  areas  aimed  at  expediting  far  more  timely  response  to  emerging 
threats,  three  of  which  are  briefly  summarized  below. 


17  This  stereo-camera  vision  system  was  developed  by  the  NASA  Jet  Propulsion  Laboratory  (JPL),  Pasadena,  CA. 
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Evolutionary  Systems  Engineering 

The  Evolutionary  Systems  Engineering  Model  for  Unmanned  Systems  shown  in  Figure  49  is  based 
on  a  wave  model  adopted  by  DoD  as  best  practices  for  Systems  of  Systems  Engineering  (SEG,  2008; 
Dahmann  et  ah,  2011;  Scrapper  et  ah,  2016;  Dove  et  ah,  2016). 18  In  2013,  this  DoD  model  was 
extended  by  the  Code  717  Unmanned  Systems  Group  to  support  the  integration,  test,  and 
experimentation  of  autonomous  systems  (Scrapper,  2014).  The  result  was  a  continuous  improvement 
process  for:  1)  assessing  system  capabilities  and  limitations;  2)  maturing  needed  technologies  based 
on  key  performance  parameters;  and,  3)  reducing  project  risk  by  understanding  performance 
tradeoffs  and  associated  costs  as  the  system  evolves. 
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Figure  49.  Developed  by  the  Code  717  Unmanned  Systems  Group  under  the  direction  of  Chris 
Scrapper,  the  Evolutionary  Systems  Engineering  Model  for  Unmanned  Systems  defines  a 
systematic  process  and  overarching  strategy  for  insertion  of  new  technology,  accumulation  of 
evidentiary  information,  and  management  of  risk. 


The  strategy  chart  shown  in  Figure  50  illustrates  all  major  processes  and  milestones  for  the 
development,  assessment,  and  integration  of  new  and  maturing  technologies  into  the  baseline 
autonomous  system.  Each  technology  that  is  introduced  must  follow  a  systematic  approach  for 
diagonal  progression  from  conception  to  validation.  The  model  partitions  a  wave  (diagonal  green 
arrows  leading  to  incremental  production  releases  along  timeline  at  bottom)  into  four  overlapping 
phases:  1)  conduct  analysis,  2)  evolve  system  architecture,  3)  integrate  capability  enhancements,  and 
4)  validate  system.  This  wave  approach  decouples  the  maturation  and  integration  processes,  and 
expedites  incremental  delivery  of  evolving  capabilities. 


18  The  concept  of  “Wave  Planning”  was  developed  by  Dr.  David  Dombkins  (2007). 
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Figure  50.  Wave-model-based  Evolutionary  Systems  Engineering  Strategy,  with  waves  illustrated 
here  by  dashed  red  diagonals  leading  to  incremental  production  releases  across  timeline  at  bottom. 

This  decoupling  allows  successive  iterations  of  increasing  capability  to  spin  off  in  the  form  of 
periodic  production  releases  for  user  evaluation  and  potential  fielding,  while  parallel  efforts  continue 
to  address  the  more  difficult  challenges.  The  previously  discussed  TeleOperated  Vehicle  project, 
initiated  in  1985  by  Code  53 1  at  the  Hawaii  Lab  and  later  handed  off  to  Code  535  in  San  Diego  for 
operational  evaluation,  could  have  significantly  benefitted  from  such  a  wave  approach.  Instead,  when 
the  exploratory  concept-development  prototype  proved  insufficiently  robust  for  rugged  USMC  field 
testing  and  demonstrations,  a  new  version  had  to  be  designed  and  built  from  the  ground  up. 19 

Standardized  Test  Methods 

Understanding  the  performance  trade-offs  for  a  variety  of  complex  and  evolving  unmanned- 
systems  configurations,  then  verifying  that  potential  solutions  can  meet  the  performance  objectives 
for  a  given  mission  set,  are  growing  problems  for  DoD  program  managers.  The  lack  of  consistent  test 
methods  and  objective  evaluation  procedures  has  seriously  hampered  users,  developers,  and  program 
managers  in  comparing  alternatives,  understanding  trade-offs  between  different  configurations,  and 
verifying  that  unmanned  systems  are  meeting  the  prescribed  performance  objectives  for  a  given 
mission  set. 

Standardized  test  methods  are  therefore  key  to  expediting  the  achievement  of  technology  readiness 
levels  (TRLs)  suitable  for  timely  fielding  of  such  systems  in  their  respective  operational  domains,  for 
the  following  reasons: 

•  Program  managers  can  use  the  test  methods  to  clearly  articulate  program  goals  in  terms  of 
desired  system  capabilities,  to  encourage  innovation,  and  to  periodically  measure  outcomes. 

•  Developers  gain  an  understanding  of  specific  robotic  capabilities  needed  by  various  user 
groups  through  Standard  Test  Method  apparatuses,  then  use  them  to  practice  and  refine  robot 
designs. 

•  Warfighters  benefit  from  the  expedited  deployment  of  unmanned  systems  that  are  cheaper, 
more  effective,  and  more  reliable. 


19  See  further  discussion  of  this  case  study  under  the  Project  Summaries  section,  this  volume. 
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The  Unmanned  Systems  Group  has  worked  closely  with  the  National  Institute  for  Standards  and 
Technology  (NIST)  to  adopt  their  standardized  test  methods  for  unmanned  systems,  and  have  NIST- 
certified  test  fixtures  in  house,  but  insufficient  space  to  set  up  and  use  them.  The  new  “Unmanned 
Systems  Integration,  Test,  and  Experimentation  facility”  (Building  585)  will  support  development  of 
standardized  test  methods  with  appropriate  test  instrumentation,  enabling  far  more  effective  inter¬ 
comparison  of  unmanned-systems  performance.  (See  later  discussion  under  the  Facilities  subsection.) 


Rapid  Prototyping 


In  2013,  at  the  initiative  of  senior  mechanical  engineer  Kurt  Talke,  the  machine  shop  was  upgraded 
to  a  Rapid  Analysis  Design  and  Prototype  Center  (Figure  51),  with  funding  provided  by  the  Naval 
Innovative  Science  and  Engineering  (NISE)  program  and  other  sources.  Some  of  the  more  significant 
equipment  procured  under  this  effort  included  the  following: 


Vertical  machining  center  (Hurco  VM10 ) 
CNC  lathe  (Trak  1 630SX) 

Waterjet  cutter  (Flow  Mach2 ) 

Knee  mill  (TrakKi) 

Environmental  chamber  (Sub-Zero  ZP-64) 


3D  printer  (Stratasys  Fortus  250MC) 
3D  laser  scanner  (Romer  7  Axis) 

Laser  engraver  (Epilog  Laser  Mini-24 ) 
Digital  force-measurement  machine 
PCB  manufacturing  oven 


Figure  51 :  Examples  of  the  new  rapid-prototyping  equipment  in  productive  use  include  the 
Southwestern  Industries  Trak  1630SX lathe  (upper  left),  Hurco  VM10  vertical  machining  center 
(bottom  left),  the  VM10  cutting  aluminum  (center),  and  the  TRAK  lathe  in  action  (right). 

This  state-of-the-art  computer-numerically-controlled  (CNC)  equipment  greatly  expedited 
mechanical  design  and  fabrication  over  multiple  prototype  iterations,  allowing  projects  to  be 
completed  in  a  much  faster  and  more  efficient  manner.  Mechanical  parts  and/or  assemblies  can  now 
be  fabricated  in-house  as  opposed  to  vended  out,  reducing  costs  and  long  lead  times  associated  with 
conventional  procurement.  In  the  past,  project  engineers  had  to  design  each  part  to  perfection  before 
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submitting  for  fabrication,  because  commercial  machine  shops  would  take  a  long  time  and  charge  a 
lot  of  money,  all  of  which  would  be  wasted  in  the  event  of  a  mistake.  The  ability  to  quickly  3D  print 
a  plastic  prototype  for  a  sanity  check  prior  to  final  fabrication  in  metal  is  a  huge  advantage  (Figure 
52). 


CAD  model 


Simulation  &  analysis 


o 

3D-printed  prototype 


Prototype  installed 
and  in  use 


Figure  52.  This  typical  project  design,  analysis,  and  development  cycle  significantly  reduced  both 
time  and  cost,  facilitating  a  more  rapid  and  effective  response  to  emergent  warfighter  needs  while 
fighting  two  wars  where  the  threat  could  change  weekly. 

A  few  representative  project  examples  enabled  by  this  new  facility  are  outlined  below,  each  of 
which  is  further  discussed  under  Project  Summaries  in  Volume  3: 

Retrotraverse  Payload  -  Developed  for  NAVEODTECHDIV  in  response  to  JUONS- CC0333  for 
use  on  both  the  iRobot  PackBot  and  QinetiQ  NA  Talon  EOD  robots,  this  add-on  autonomy  module 
supported  the  following: 

•  Retrotraverse  upon  lost  communication 

•  Retrotraverse  to  a  predetermined  location 

•  Waypoint  navigation 

•  Leader- follower  behavior 

•  Follow-the-OCU  behavior 

•  Framework  for  implementation  of  other  advanced  behaviors 

Manually  Deployed  Communications  Relays  -  Also  developed  for  NAVEODTECHDIV  in 
response  to  .JU CWS-CC04 1 2  for  use  on  both  the  PackBot  and  Talon  EOD  robots,  this  quick-turn- 
around  effort  provided  communication  mesh  networking  using  all  available  RF  nodes.  This  approach 
allowed  PackBots  and  Talons  to  relay  for  one  another  and/or  to  use  each  other’s  relay  nodes, 
resulting  in  more  CONOPS  flexibility  with  greater  operator  standoff. 

RoboZap  -  Developed  in  support  of  the  U.S.  Army  Rapid  Equipping  Force  (REF)  for  neutralizing 
improvised  explosive  devices,  this  system  was  based  upon  a  commercial  BobCat  vehicle  outfitted 
with  a  QuinetiQ  NA  applique  kit  for  teleoperation.  The  rear-mounted  payload  was  a  high-voltage 
generator  developed  by  Xtreme  Alternative  Defense  Systems  (XADS)  that  charged  two  electrodes 
trailing  over  the  ground,  which  collectively  introduced  a  decapacitating  current  spike  in  buried  IED 
command  wires 

On  the  electronics  side,  the  unmanned  systems  group  uses  Altium  Designer  as  the  printed  circuit 
board  (PCB)  design  tool,  which  offers  schematic  capture,  PCB  layout,  SPICE  simulator  and  signal 
integrity  analysis.  To  test  specific  functions  of  a  larger  circuit,  we  are  capable  of  prototyping  two- 
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layer  PCBs  in  house  using  the  LPKF  S63  plotter  (Figure  53).  This  plotter  is  capable  of  creating  small 
geometries,  such  as  0. 1  mm  (4  mil)  track  width/separation  and  pads  for  0201  (0.6  mm  x  0.3  mm) 
components.  With  a  10-tool  auto-exchanger,  the  plotter  finishes  a  single  side  without  supervision, 
requiring  only  a  manual  flip  of  the  copper  clad  to  finish  the  second  side.  PCB  assembly  is  for  the 
most  part  executed  manually. 


Rapid  PCB  Prototyping 

■  LPKF  S63  plotter  with  auto  tool  exchange 

■  Double-sided  PCBs  (upgradable  to  8  layers) 

■  Milling  down  to  0.1  mm  thick  traces 

■  Milling  for  0201  components  (0.6  x  0.3  mm) 

PCB  Assembly  and  Rework 

■  Precision  component  placement  using  PL550  (e  g., 
ball-grid  array  package) 

■  Component  soldering  of  bottom-terminated 
components  like  BGAs  and  components  with  thermal 
tab  using  infrared  soldering  (I  R)  station  IR550 

■  Depopulating  (desolderinq)  components  with  IR550 

High  Power  Microscope 

■  x7  to  x40  magnification  Lynx  microscope 

■  LED  ring  illumination  .  ■ 

■  Used  for  manual  placement  and  hand- 
soldering  of  components 

-  0201  components 

-  0.5  mm  pitch  packages  L,nx  Microscope 


S63  Plotter 


r/ga 

S63  Tool  Exchange 


IR550 


obMS  . 


PL550 


Figure  53.  Some  of  the  equipment  used  in  our  advanced  printed-circuit-board  fabrication  lab  in 
Building  622. 


The  use  of  a  Vision  Engineering  high-power  Lynx  Elite  microscope  with  adjustable  magnification 
(between  7x  and  40x)  allows  0201  components  and  integrated  circuits  with  0.5-millimeter  pitch  to  be 
soldered  by  hand.  This  microscope  also  facilitates  inspection  of  boards  for  correct  assembly  and 
identification  of  potentially  damaged  components  not  visible  to  the  naked  eye.  Bottom-terminated 
components  (BTCs)  such  as  ball-grid  arrays  or  components  with  thermal  tabs,  which  cannot  be 
manually  assembled,  are  handled  by  a  Kurtz  Ersa  PL550  pick-and-place  machine  and  IR550  infrared 
soldering  station.  The  pick-and-place  machine  allows  application  of  solder  paste  to  BTCs,  followed 
by  precision  placement  onto  the  PCB.  The  IR550  is  then  used  to  solder  the  component  using  the 
appropriate  heat  profile.  For  rework,  the  IR550  can  also  be  used  to  depopulated  components  without 
damaging  the  PCB. 
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Figure  54.  Electrical  engineer  Daniel  Leung  inspects  the  final  assembly  of  a  ModKit  Electronics 
Stack  completed  by  Yuong  Sun  at  the  advanced  printed  circuit  board  work  center  in  Building  622. 

FACILITIES 

As  the  Adaptive  Systems  Branch’s  1986  Seaside  accommodations  consisted  of  only  three 
dilapidated  trailers  and  an  empty  shell  of  a  warehouse  with  no  heat  or  hot  water  (Figure  55),  there 
was  a  serious  lack  of  adequate  workspace  for  effective  execution  of  unmanned  systems  development. 
Since  the  warehouse  structure  was  not  climate  controlled,  significant  corrosion  problems  were 
experienced  with  the  electronic  circuit  boards  in  the  robots,  as  well  as  the  associated  tools  and  test 
equipment  used  in  the  space.  These  major  shortcomings  adversely  affected  not  only  the  branch’s 
efficiency,  but  more  importantly,  the  perception  of  its  potential  capabilities  by  visiting  sponsors  who 
were  expecting  to  see  a  state-of-the-art  facility. 


Figure  55.  a)  Building  F-36,  surrounded  by  aging  trailers  at  upper  right,  as  shown  looking  east  in 
1988.  Buildings  622  and  624  would  later  be  constructed  in  the  open  area  this  side  of  Woodward 
Road  at  upper  left  (see  Figure  57).  b)  Electrical  engineer  Gary  Gilbreath  in  Building  F-36,  which 
lacked  heat,  hot  water,  and  partitions,  circa  1987. 
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Commander  Bart  Everett,  who  ran  a  part-time  construction  company  to  supplement  his  Navy 
scholarship  at  Georgia  Tech  in  the  early  1970s,  began  working  nights  and  weekends  in  1988  to  outfit 
Building  F-36  with  partitions  and  ceilings  (Figure  56).  As  this  was  the  branch’s  only  building  at  the 
time,  the  upgrade  was  conducted  in  four  successive  phases  to  allow  work  to  continue  on  the  USMC 
Teleoperated  Vehicle  (TOY)  project  during  the  renovation.  The  result  was  a  far  more  suitable 
environment  for  software  development,  with  a  new  shower-equipped  bathroom,  a  conference  room,  a 
separate  technician’s  space,  a  lab  area,  and  an  open  bay  in  the  center  for  testing  the  ROBART II, 
ModBot,  and  MDARS  indoor  robots. 


Figure  56.  a)  Over  a  period  of  approximately  4  months,  Commander  Everett  partitioned  Building 
F-36  on  his  own  time  in  stages:  rooms  101  and  102  at  left,  rooms  104-107  at  the  north  end 
(background),  and  rooms  108  and  109  at  right,  b)  Todd  Everett  assisted  his  dad  on  weekends 
with  the  final  ceiling-joist  installation  over  the  central  bay. 

The  terrain  along  Woodward  and  Gatchell  Roads  was  scouted  for  outdoor  test  sites  for  the  TOV 
project,  with  the  subsequent  selection  of  the  Battery  Gillespie  area  just  south  of  the  northern  fence 
line  and  west  of  Woodward  Road  (Figure  57a).  A  gate  in  this  fence  could  be  opened  to  allow 
expanded  runs  along  Woodward  Road  through  adjoining  Navy  property  to  the  north  (Figure  57b). 
The  Surrogate  Teleoperated  Vehicle  (STV)  follow-on  to  TOV  also  underwent  off-road  testing  at  this 
site  in  1992.  The  following  year,  the  MDARS-Exterior  dune-buggy  surrogate  was  demonstrated  here 
to  convince  the  sponsor  that  retrofitting  an  existing  vehicle  was  problematic.20  Numerous  projects 
use  this  Northern  Test  Site  today  for  readily  available  outdoor  evaluation. 


20 


With  the  more  recent  introduction  of  commercial  drive-by-wire  systems,  the  philosophy  today  is  just  the  opposite. 
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a) 


b) 


Figure  57.  a)  The  Unmanned  Systems  Group  campus  along  the  Pacific  Ocean  on  Woodward  Road, 
circa  October  2015  (satellite  image  courtesy  Google  Maps),  b)  The  Northern  Test  Site  in  the  Battery 
Gillespie  area  was  extensively  used  during  testing  of  the  USMC  Ground  Air  TeleRobotic  Systems 
(GATERS)  project,  circa  1988. 

In  1991,  Commander  Everett  constructed  a  north-wing  addition  to  Building  F-36  to  house  a  small 
machine  shop  that  would  better  support  emergent  requests  for  quick-turnaround  solutions  to  user 
needs  (Figure  58).  The  WWII-era  Bridgeport  mill  that  had  occupied  a  small  comer  of  the  F-36  bay 
was  relocated  to  this  addition  and  complemented  with  surplus  equipment  rounded  up  from  various 
Center  sources,  including  a  drill  press,  band  saw,  sheet-metal  sheer,  and  forming  brake.  To  provide 
heat  during  the  winter  and  protect  the  Unmanned  Systems  Branch’s  growing  collection  of  expensive 
equipment  from  the  corrosive  salt-laden  Pacific  Ocean  mist,  Seaside  Facilities  Manager  Tom  Gaydos 
arranged  for  the  entire  building  to  be  air  conditioned. 


Figure  58.  Commander  Everett  added  the  north  wing  machine  shop  lean-to  addition  (behind  car) 
to  Building  F-36  in  1 991 ,  which  was  later  expanded  westward  (to  right  of  windows)  as  part  of  the 
contracted  south-wing  addition  to  the  building  (extreme  left)  in  2010. 

The  GATORS/TOV project  employed  a  number  of  HMMWV- based  vehicles  (two  as  UGVs,  two 
equipped  with  command  and  control  shelters,  and  one  for  cable  recovery),  which  seriously  overtaxed 
the  limited  capacity  of  F-36.  In  response  to  this  need,  Seaside  Facilities  Manager  Tom  Gaydos 
offered  up  two  prefabricated  Vietnam-era  “Butler  Huts”  stored  in  wooden  crates  just  north  of  the  San 
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Diego  Sewer  Plant.  These  galvanized-steel  kits  were  quickly  erected  by  branch  personnel  (on 
concrete  slabs  poured  by  Facilities)  as  Buildings  608  and  618  (Figure  59).  This  identical  pair  of  20- 
foot  by  48-foot  climate-controlled  structures  added  much  needed  indoor  work  areas  for  large 
unmanned  vehicles  at  virtually  no  cost. 


Figure  59.  a)  Bart  Everett  and  Tracy  Heath-Pastore  begin  assembling  the  steel  framework  for 
Building  618  in  1993.  A  new  parking  lot  was  later  constructed  on  the  east  side  of  this  structure  along 
Woodward  Road,  b)  Building  608,  circa  2015,  was  similarly  erected  by  Adaptive  Systems  Branch 
personnel  southeast  of  Building  F-33  (see  again  Figure  57). 


Following  relocation  of  the  former  occupant  at  the  end  of  the  Cold  War  in  1992,  an  indoor  robotic 
test  facility  was  created  inside  Battery  Woodward  (Building  F-12),  a  200-Series  WWII  coast-defense 
gun  battery.  Almost  all  of  the  Center’s  indoor  UGV  programs  conducted  some  form  of  testing  over 
the  years  in  this  15-room  underground  bunker  (Figure  60),  as  further  discussed  in  the  Project 
Summaries  section.  Additional  outdoor  test  areas  were  set  up  just  north  of  this  site  for  tethered 
testing  of  vertical-takeoff-and-landing  air  vehicles,  along  with  a  ffee-flight  launch  pad  and  covered- 
slab  area  for  test  personnel  and  observers.  A  large  tilt  table  designed  by  lead  mechanical  engineer 
Aaron  Burmeister  was  erected  for  man-portable  UGV  stability  tests. 


Figure  60.  a)  The  MDARS-Interior  robot  undergoes  testing  in  Battery  Woodward,  circa  2000. 
b)  ROBART  III  finds  its  designated  target  (soda  can  in  brown  box)  after  searching  the 
underground  bunker,  circa  2004.  c)  An  iRobot  PackBot  enters  Battery  Woodward  during  tests  of 
the  communications  relay  deployer  module,  circa  2007. 
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To  provide  a  more  realistic  set  of  navigational  path  segments  for  the  MDARS-Interior  robot 
development  in  Building  F-36,  a  6-foot-wide  concrete  apron  was  installed  along  the  south  and  east 
sides  of  the  structure  as  shown  in  Figure  61a.  This  simple  but  effective  upgrade  allowed  the  robot  to 
exit  the  building  interior  through  the  roll-up  garage  door  and  patrol  outdoors,  effectively  doubling  the 
available  route-segment  footage  at  minimal  cost  almost  overnight.  Once  again,  we  greatly  benefitted 
from  the  positive  can-do  attitude  of  our  Seaside  Facilities  Manager  Tom  Gaydos  (Figure  61b),  who 
unfortunately  retired  a  short  time  later. 


Figure  61 .  a)  A  6-foot-wide  concrete  apron  was  added  around  F-36  to  provide  more  complex 
pathways  for  testing  the  MDARS-Interior  robot,  b)  Commander  Bart  Everett  (left)  delivers  a  heartfelt 
thank-you  speech  for  Seaside  Facilities  Manager  Tom  Gaydos  during  the  latter’s  potluck  retirement 
picnic  at  the  volleyball  court. 

In  2001,  Building  F-33  overlooking  the  Pacific  was  remodeled  to  replace  the  leaky  windows, 
remove  the  large  Faraday  cage  inside,  and  demolish  the  makeshift  radar  equipment  room  at  bottom 
right  in  Figure  62a.  Given  its  close  proximity  to  the  sea,  a  central  air  conditioning  system  was 
installed  to  provide  a  climate-controlled  environment  for  five  new  lab  and  office  spaces.  An  extra 
wide  entry  hallway  with  a  glass  door  and  windows  facilitated  test  and  evaluation  of  man-portable 
exterior  robots.  The  test  engineer  could  set  up  as  shown  in  Figure  62b  for  good  visibility  of  the  robot 
operating  in  the  adjacent  parking  lot  and  along  Robart  Road. 


Figure  62.  a)  Shown  looking  northwest,  circa  1987,  Building  F-33  was  originally  occupied  by  another 
code  performing  high-resolution  radar  research,  b)  This  2002  photo  shows  engineer  Jeff 
Muehlhauser  performing  outdoor  testing  of  the  man-portable  URBOT from  the  remodeled  entryway 
of  F-33. 
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To  further  accommodate  our  growing  staff,  construction  of  the  ground  floor  of  Building  622  began 
in  the  spring  of  1992,  soon  followed  by  Building  624  just  to  the  east  (Figure  63).  Each  of  these 
identical  structures  had  1 1  rooms,  including  a  large  lab  area  and  a  conference  room.  As  more 
engineers  were  brought  on  board  to  support  the  steadily  increasing  number  of  projects,  Commander 
Everett  partitioned  off  a  few  of  the  larger  rooms  to  provide  additional  offices.  In  anticipation  of  the 
growing  need  for  space,  both  these  buildings  were  designed  and  built  to  support  a  subsequent 
second-story  addition. 


Figure  63.  Newly  constructed  Buildings  622  (grey  roof)  and  624  (red  roof)  are  shown  at  upper  left  in 
this  aerial  view  of  the  growing  unmanned  systems  campus  overlooking  the  Pacific  Ocean  on 
Woodward  Road  looking  east,  circa  1994.  Note  sand  volleyball  court  just  to  the  south  at  center  right. 

In  the  spring  of  2004,  construction  began  on  the  planned  second-floor  upgrade  to  Buildings  622 
(Figure  64),  complete  with  20  new  office  spaces,  a  securable  storage  area,  and  an  additional 
restroom.  In  2007,  a  similar  second-floor  addition  was  added  to  Building  624  next  door,  with 
10  offices,  a  restroom,  and  a  dedicated  server  room.  Initially  left  open  for  use  as  an  indoor-robot 
assembly  and  test  area,  the  remaining  square  footage  was  eventually  replaced  by  a  six-cell  cube  farm 
to  accommodate  New  Professionals  and  summer  interns.  By  this  time,  the  Unmanned  Systems 
Branch  had  reached  critical  mass  for  office  and  lab  space,  and  the  focus  shifted  to  the  facilitation  of 
unmanned  ground  vehicle  support  for  the  war  effort  in  Afghanistan  and  Iraq. 
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Figure  64.  The  second-story  addition  to  Building  622  is  shown  midway  through  the  framing  phase, 
circa  April  2004,  with  the  single-story  Building  624  at  background  right,  b)  Buildings  622  and  624  in 
2015.  The  WWII-era  switchback  roads  on  the  hill  at  upper  right  were  earlier  used  for  dune-buggy 
testing. 

New  construction  began  in  2006  for  Building  594,  a  3000-square-foot  structure  for  medium  and 
large  unmanned- vehicle  development.  The  highlight  of  the  grand-opening  ceremony  on  1  May  2007 
was  when  Commander  Mike  McMillan,  executive  officer  of  Space  and  Naval  Warfare  Systems 
Center  San  Diego,  cut  the  ribbon  using  a  QinetiQ  Talon  robot  (Figure  65a).  A  rail-mounted  overhead 
crane  assisted  in  the  assembly  and  maintenance  of  large  system  components,  with  a  double  garage 
door  for  vehicle  access.  Initially  used  to  store  and  support  two  unmanned  surface  vehicles,  Building 
594  currently  serves  as  the  base  of  operations  for  the  ONR  30  Ground  System  Autonomy  project 
(Figure  65b). 


Figure  65.  a)  Commander  Mike  McMillan  (center)  operates  a  QinetiQ  Talon  robot  equipped  with 
ceremonial  shears  during  the  Building  594  ribbon-cutting  ceremony  on  1  May,  2007.  b)  The  roomy 
interior  of  the  new  high-bay  structure  would  soon  be  filled  with  a  variety  of  large  unmanned  surface 
vehicles  and  exterior  ground  robots. 
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In  2007,  the  Unmanned  Systems  Branch’s  Southern  Test  Site  for  more  rugged  3D-terrain  off-road 
testing  of  unmanned  ground  vehicles  was  established  at  the  south  end  of  Gatchell  Road  (Figure  66a), 
just  north  of  the  San  Diego  Sewer  Plant.  As  this  location  was  some  1 .2  miles  away  from  the  main 
campus,  a  prefabricated  steel  support  structure  (Building  651)  was  erected  for  electrical  power, 
telephone  service,  and  work  and  storage  areas  (Figure  66b).  This  facility  was  used  by  the  Unmanned 
Systems  Reserve  Unit  for  training  and  other  exterior  robot  testing,  and  later  by  flight-systems 
engineer  Nick  Stroumtsos  for  indoor  UAV  swarm  demonstrations.  It  currently  supports  testing  for 
the  ONR  30  Ground  System  Autonomy  project  along  Gatchell  Road  and  the  parallel  dirt  trail  to  the 
east  (see  Figure  66a). 


Figure  66.  a)  Building  651  was  constructed  on  the  Southern  Test  Site  adjacent  to  the  Pacific  Ocean 
just  north  of  the  San  Diego  Sewer  Plant  (satellite  image  courtesy  Google  Maps),  b)  Building  651  in 
2015.  The  tower  structure  in  background  was  part  of  Convair’s  Atlas  intercontinental  ballistic  missile 
test  site  in  the  1 950s. 

By  2013,  a  remodel  and  expansion  of  Building  F-36  had  refurbished  both  restrooms,  rebuilt  the 
rooftop  observation  deck,  added  a  new  south  wing  for  office  spaces,  and  extended  the  north-wing 
machine-shop  area  to  accommodate  the  expanded  CNC  rapid-prototyping  facility.  The  following 
year,  a  Capital  Investment  Plan  (CIP)  Minor  Construction  Proposal  was  submitted  for  a  6000-square- 
foot  pre-engineered  steel  structure  (60  ft.  x  100  ft.  x  25  ft.  high)  to  serve  as  an  Unmanned  Systems 
Integration,  Test,  and  Experimentation  (UxSITE)  facility.  A  concrete  pad  on  the  south  end  of  the 
building  will  accommodate  two  40-foot  ISO  containers  that  can  be  moved  into  position  in  support  of 
emergent  test  requirements. 
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Figure  67.  a)  Site  preparation  for  Building  585  began  with  demolition  of  the  foundation  for  the  oil-well 
derrick  that  supported  the  Convair  MX-774  missile  in  1949.  b)  The  MX-774  was  a  post-war  German 
V-2  derivative  that  was  the  precursor  to  the  Atlas  intercontinental  ballistic  missile,  which  was  later 
tested  north  of  the  San  Diego  sewer  plant. 

Michael  Bruch,  Tracy  Pastore,  and  Rachel  TenWolde  attended  the  Final  Design  Review  for 
Building  585  in  early  December  2015.  Site  preparation  began  later  that  same  month  west  of  Woodard 
Road  (Figure  67a),  and  the  main  slab  was  poured  on  28  January  (Figure  68a).  The  steel  support 
structure  was  completed  in  February  and  installation  of  the  exterior  panels  and  insulation  well 
underway  (Figure  68b),  with  a  projected  final  completion  date  of  June  2016.  This  new  facility  will 
support  rapid  integration  and  standardized  testing  of  sensors,  subsystems,  outdoor  vehicles, 
command-and-control  systems,  and  integrated  unmanned  systems  of  systems. 


Figure  68.  a)  The  main  slab  for  Building  585  was  poured  on  28  January  2016  at  the  Central  Test 
Site,  just  north  of  Woodward  Loop  across  from  Building  594.  b)  Building  585  construction  looking 
north  on  29  February  2016. 
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PROJECT  SUMMARIES 


Continuing  S&T  development  of  key  supporting  technologies  such  as  autonomy,  navigation, 
multi-agent  behavior,  and  common  control  is  critical  to  effective  fielding  of  robust  and  adept 
unmanned  systems.  The  modular  open-architecture  approach  employed  by  the  Code  717  Unmanned 
Systems  Group  greatly  facilitates  both  evolutionary  upgrade  and  technology  transfer,  allowing  new 
projects  to  benefit  from  prior  efforts,  most  of  which  went  through  their  own  iterative  stages  of 
continuing  enhancement. 

From  just  a  handful  of  projects  in  the  early  1980s,  the  group’s  involvement  had  progressively 
expanded  to  more  than  50  ongoing  programs  and  initiatives  at  the  end  of  FY  2014,  funded  by  various 
service  and  government  agencies.  Short  summaries  of  the  many  endeavors  across  all  operational 
domains  (air,  land,  and  sea)  are  presented  in  chronological  order  throughout  the  remainder  of  this 
document  (and  follow-on  Volumes  2  and  3),  broken  out  by  decade,  with  numerous  references  to 
related  publications  offering  greater  detail.  See  also  the  project  history  timeline  graphic  presented  in 
Appendix  F. 

CHRONOLOGICAL  PROJECT  SUMMARIES  (1970-1979) 

Remote  Unmanned  Work  System  (1970-1980) 

While  preliminary  work  leading  to  the  Remote  Unmanned  Work  System  (RUWS)  actually  began  in 
1968,  the  first  of  several  demonstration  dives  took  place  off  Oahu,  HI,  in  the  mid-1970s  (Lemaire, 
1988).  The  tethered  submersible  was  designed  to  perform  a  variety  of  underwater  tasks  at  depths  as 
great  as  20,000  feet  (Figure  69),  giving  it  access  to  98%  of  the  ocean  floor  (Talkington,  1978). 
Developed  by  NOSC  under  the  Deep  Ocean  Technology  Program,21  the  complete  RUWS  system 
consisted  of  the  Work  Vehicle  and  the  Primary  Cable  Termination  (PCT)  component  to  which  it  was 
firmly  attached  during  deployment  and  recovery. 


Figure  69.  Shown  here  at  the  NOSC  Hawaii  Laboratory  on  Kaneohe  Marine  Corps  Air  Station,  Oahu, 
the  Remote  Unmanned  Work  System  (RUWS)  Work  Vehicle  measured  4.5  by  4.5  by  1 1  feet  long, 
weighed  4300  pounds  in  air,  and  could  operate  down  to  20,000  feet  (Talkington,  1978). 


21  The  Deep  Ocean  Technology  Program  was  managed  by  the  Naval  Sea  Systems  Command  (NAVSEA  05R2). 
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Once  released  at  depth,  the  RUWS  Work  Vehicle  moved  freely  at  the  end  of  an  850-foot  neutrally 
buoyant  flexible  tether  deployed  from  the  PCT  (Figure  70a).  In  addition  to  launch  and  recovery  of 
the  RUWS  vehicle,  other  functions  of  the  PCT  were  to  maintain  tension  in  the  primary  cable  and 
provide  position-keeping  capability  for  its  bottom  end.22  A  third  key  component  was  the  Motion 
Compensating  Deck  Handling  System  (MCDHS),  a  versatile  combination  of  launch  crane  and  cable 
reel  (Figure  70b).  Its  purpose  was  to  launch  and  recover  the  RUWS! PCT  combination  and  tend  the 
primary  cable,  minimizing  ship-induced  motion  and  tension  therein. 


b) 

Figure  70.  a)  The  RUWS  Work  Vehicle  (bottom  right)  moved  freely  at  the  end  of  a  buoyant  flexible 
tether  deployed  from  the  Primary  Cable  Termination  (PCT)  component  (upper  left),  b)  The  MCDHS 
with  RUWS  attached  beneath  the  PCT  could  operate  in  sea-state  three  and  perform  recovery  in  sea- 
state  four. 

A  seven-function  hydraulic  master-slave  arm  was  terminus  controlled  by  the  operator  moving  a 
simple  pistol  grip  to  vary  the  position  and  orientation  of  the  remote  manipulator  (Figure  71a).  Force- 
feedback  allowed  any  resistance  encountered  by  the  remote  arm  to  be  reproduced  at  a  scaled-down 
level  in  the  master  arm,  and  thus  felt  by  the  operator.  A  second  simpler  and  more  rugged  four- 
function  gripper  arm  was  used  to  steady  R UWS  to  the  workpiece  or  worksite.  An  artist’s  concept  of 
the  Work  System  Package  (WSP)  mounted  on  the  manned  submersible  Alvin  is  shown  in  Figure  71b 
(Wemli,  1979). 


22  In  effect,  the  PCT  was  a  sophisticated  plumb  bob,  keeping  the  main  umbilical  section  vertical  and  isolating  the 
RUWS  work  vehicle  from  its  motions. 
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Figure  71.  a)  In  this  land-based  training  mockup,  Jim  Held  (background)  manipulates  the  7-function 
master  input  device  to  control  the  slave  (center),  with  the  more  simplistic  gripper  arm  used  for 
workpiece  stabilization  at  foreground  right,  b)  The  WSP  as  it  would  appear  mounted  on  the  manned 
submersible  Alvin  (Wernli,  1979). 

Following  6  weeks  of  operational  testing  with  the  Cable  Controlled  Underwater  Recovery  Vehicle 
(CURVIII)  at  San  Clemente  Island  off  California  in  1976  (Wernli,  1980), 23  the  WSP  application 
payload  was  flown  to  the  NOSC  Hawaii  Laboratory  for  integration  onto  RUWS.  Further  test  pool 
evaluation  followed  in  preparation  for  supporting  the  upcoming  Large  Object  Salvage  System  (LOSS) 
operational  demonstration  at  the  Naval  Coastal  Systems  Center  (NCSC).  The  WSP/RUWS 
combination  was  then  flown  to  Panama  City,  FL,  where  it  successfully  supported  LOSS 
demonstrations  during  1976  and  1977  (Wernli,  1979). 


Figure  72.  After  recovering  wreckage  from  the  Space  Shuttle  Challenger  disaster  in  January  1986, 
CURV  III  was  transferred  to  the  Navy  Supervisor  of  Salvage  (SUPSALV)  and  upgraded  for 
operations  down  to  20,000  feet.  The  rugged  and  enduring  unmanned  undersea  system  was  still 
operational  almost  30  years  later. 


23  A  full  account  of  this  testing  is  provided  in  NOSC  Technical  Report  553,  available  from  the  Defense  Technical 
Information  Center  (DTIC)  (Wernli,  1980). 
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The  WSP  payload  was  then  returned  to  San  Diego  where  it  underwent  an  extensive  laboratory 
evaluation  of  the  operating  characteristics,  resulting  in  a  complete  time-motion  baseline  description 
of  the  system.  The  payload  was  further  studied  for  integration  onto  the  Pontoon  Implacement  Vehicle 
(PIV)  for  deep  ocean  recovery  (Wemli,  1979).  Ironically,  the  RUWS  work  vehicle  was  lost  at  sea  in 
January  1980  due  to  a  design  flaw  in  its  mechanical  attachment  to  the  PCT,  and  subsequent  search 
and  recovery  efforts  proved  futile  due  to  the  extremely  rough  scarp  bottom. 

CHRONOLOGICAL  PROJECT  SUMMARIES  (1980-1989) 

ROBARTItf 980-1 982) 

One  of  the  very  first  behavior-based  autonomous  robots  (Figure  73a),  ROB  ART  l  was  then 
Lieutenant  Commander  Bart  Everett's  thesis  project  at  the  Naval  Postgraduate  School  (NPS)  in 
Monterey,  CA  (Everett,  1982a;  1982b).  This  robotic  security  system  was  rather  unique  at  the  time  in 
that  it  was  fully  autonomous  with  no  RF  link  or  operator  control  unit  (OCU);  human  input  was 
normally  limited  to  a  single  toggle  switch  that  enabled/disabled  the  Security  Assessment  mode 
(Figure  73b).  If  needed,  a  small  troubleshooting  panel  with  four  toggle  switches  and  a  pushbutton  for 
speech-prompted  binary  entry  of  two  four-bit  nibbles  could  be  used  to  invoke  various  diagnostic 
routines  (Figure  73b).  Output  responses  were  conveyed  via  speech  synthesis  using  the  newly 
introduced  National  Semiconductor  DigiTalker  board. 


Figure  73.  a)  Lieutenant  Commander  Bart  Everett,  USN,  poses  with  ROBART I  during  filming  for 
ABC  World  News  Tonight  while  attending  the  Naval  Postgraduate  School  in  Monterey,  CA,  circa 
1982.  b)  The  four-bit  troubleshooting  panel  is  on  the  left  side  of  the  robot  at  bottom  center. 

The  robot’s  mission  was  to  patrol  a  home  environment,  following  either  a  random  or  somewhat 
predetermined  pattern  from  room  to  room,  checking  for  unwanted  conditions  such  as  fire,  smoke, 
flooding,  or  intrusion.  The  security  application  was  chosen  because  it  demonstrated  performance  of  a 
useful  function  that  did  not  require  an  end-effector  or  vision  system.  Patrols  were  made  at  random 
intervals,  with  the  majority  of  time  spent  immobile  in  Security  Assessment  Mode  to  conserve  power 
and  enable  detection,  as  the  on-board  motion  sensors  could  only  work  properly  when  the  robotic 
platform  was  not  moving  (Everett,  1980). 
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One  exception  was  the  head-mounted  passive-infrared  (PIR)  motion  detector  made  by  Colorado 
Electro-Optics,  which  had  a  maximum  range  of  about  50  feet  indoors  (Everett,  1982b): 

“This  unit  is  intended  to  be  mounted  in  a  stationary  position,  but  was 
found  to  be  stable  enough  in  an  average  household-temperature 
environment  to  operate  when  the  vehicle  was  in  motion,  due  to  the  low 
speed  of  advance ...” 

“If  the  head  is  turned  to  one  side  as  the  vehicle  is  moving,  the  presence  of 
an  intruder  on  that  side  will  be  detected  due  to  relative  motion  with 
respect  to  the  vehicle.  An  obvious  advantage  would  therefore  be  realized 
by  mounting  a  detector  on  each  side  of  the  vehicle,  in  addition  to  the  one 
mounted  on  the  head.” 

This  demonstration  of  human-presence  detection  from  a  moving  robot,  which  for  years  since  has 
been  the  Holy  Grail  of  physical-security  applications,  worked  only  indoors  at  very  slow  speed.24 

In  addition  to  the  above  PIR,  optical,  ultrasonic,  and  hearing  sensors  were  used  to  detect  intruder 
motion,  with  other  sensors  monitoring  for  vibration,  fire,  smoke,  toxic  gas,  and  flooding  (Everett, 
1982a).  Some  of  these  inputs  were  hard-wired  to  cause  a  security  alert,  whereas  others  had  to  be 
evaluated  first  by  software  that  could  then  trigger  an  alert  if  warranted.  Operational  feedback 
regarding  the  pros  and  cons  of  this  approach  led  to  the  vastly  improved  security-assessment 
algorithm  later  developed  on  ROBART II,  as  alluded  to  at  the  time  by  Everett  (1982b): 

“As  the  software  is  developed  for  the  intrusion-detection  scheme 
employed  by  the  robot,  it  soon  becomes  apparent  that  in  most 
environments  confirming  information  from  other  sensors  should  be 
obtained  to  minimize  false  alarms.” 


Figure  74.  Intruder-detection  sensors  on  ROBART  I  included  a  passive-infrared  motion  sensor  at  the 
very  top  of  the  head,  three  optical  motion  sensors  immediately  above  the  beacon-tracking 
collimating  tubes  on  the  front  face,  two  microphones  for  ears,  and  a  vibration  sensor  in  the  torso  (not 
shown). 


24  Note  that  this  configuration  would  detect  any  discreet  thermal  source  such  as  a  stove  burner,  and  not  just  humans. 
See  chronological  Project  Summaries  section  in  Volume  2  for  Human  Presence  Detection  and  Human  Presence 
Detection  and  Assessment. 
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ROB  ART  I  was  powered  by  a  12-volt,  20-amphour  lead-acid  battery  that  gave  about  6  hours  of 
continuous  service  before  initiation  of  automatic  recharging.  A  1-hour  power  reserve  was  allocated 
for  locating  an  optical  homing  beacon  atop  the  charging  station  after  the  battery-monitor  circuit 
detected  a  low- voltage  condition.  The  origins  of  this  phototropic  behavior  dated  back  to  the  Crawler 
(Figure  75a),  an  autonomous  tracked  robot  Everett  built  in  1965  as  a  high-school  science-fair  project 
(Everett,  1995a;  Davis,  201 1).25  The  rotating  collimating-tube  orientation  relative  to  vehicle  front- 
center  whenever  the  beacon  was  sensed  by  the  tube  photocell  determined  which  drive  track  to  stop 
and  for  how  long  during  each  scan  revolution  (Figure  75b). 


Figure  75.  a)  CRAWLER  I,  circa  1965,  was  an  autonomous  tracked  robot  with  a  rotating  photocell 
scanner  that  enabled  a  phototropic  homing  response  for  recharging,  b)  Once  beacon  detection  by 
the  photocell  coincided  with  the  collimating  tube  passing  through  vehicle  front-center  as  shown, 
corrective  steering  was  inhibited  with  the  robot  pointed  directly  at  the  charger. 

The  improved  beacon-tracking  system  on  ROBART I  some  16  years  later  employed  a  three- 
element  photocell  array  to  automatically  keep  the  head  pointed  at  the  beacon  (Figure  76a),  so  the 
head  pan  angle  could  be  used  to  position  the  steerable  front-wheel  for  homing.  The  free-standing 
recharger  beacon  was  initially  located  by  panning  the  head  through  its  full  extent  of  travel  while 
digitizing  the  perceived  light  level  as  a  function  of  the  pan-axis  encoder  value.  The  optical  beacon  on 
the  charger  was  then  activated  using  a  garage-door-type  RF  link,  after  which  another  full  scan  was 
performed.  Assuming  the  recharging  station  was  within  the  surveyed  area  of  regard,  subtracting  these 
two  linear  arrays  yielded  a  peak  intensity  differential  corresponding  to  the  beacon’s  relative  bearing 
(Figure  76b). 


25  The  Crawler  robot  was  intended  for  entry  in  the  1967  science  fair,  unfortunately  the  same  year  this  long-standing 
annual  event  was  discontinued  by  the  faculty  at  Moultrie  High  School  in  Mt.  Pleasant,  SC. 
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Figure  76.  a)  This  early  1981  version  of  ROBART I  has  activated  the  optical  beacon  on  its 
recharging  station  and  is  preparing  to  dock,  b)  Analog  photocell  output  as  a  function  of  scan  angle, 
c)  Shown  here  several  months  later  with  the  head  covered,  the  robot  has  docked  with  the  recharging 
station,  but  not  yet  deactivated  the  optical  beacon. 

If  no  beacon  was  sensed,  the  robot  turned  180  degrees  to  repeat  the  scanning  process.  Once  beacon 
acquisition  was  confirmed,  the  Dock  behavior  (Table  2)  slaved  the  front- wheel  steering  actuator  to 
the  head  position  during  forward  travel.  The  metal  pole  supporting  the  homing  beacon  served  as  the 
ground  (GND)  contact  for  the  charging  circuit,  its  mating  surface  being  the  spring-loaded  front- 
bumper  panel.  The  connection  for  the  HOT  leg  was  through  a  circular  aluminum  plate  at  the  base  of 
the  beacon  tower,  which  was  electrically  isolated  from  the  vertical  pole  by  a  Plexiglas  insulator  (see 
Figure  76a).  The  spring  probes  that  mated  with  this  plate  extended  downward  from  a  small  plastic 
box  attached  to  the  drive-wheel  support  cage  (Figure  77). 


Figure  77.  Tandem  drive  motors  were  attached  to  each  side  of  the  front  wheel  for  forward  and 
reverse  motion,  while  a  third  motor  situated  above  provided  ±80  degrees  of  steering  angle.  The 
three  springs  descending  from  the  small  black  box  attached  to  the  drive-wheel  assembly  made 
contact  with  the  circular  plate  on  the  recharger  base. 

ROBART I’s  autonomous-navigation  scheme  featured  a  layered  hierarchy  of  behaviors  (Table  2) 
that  looked  ahead  for  a  clear  path  (high-level),  reactively  avoided  nearby  obstacles  (intermediate- 
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level),  and  responded  to  actual  impacts  (low-level).  A  basic  tenet  of  this  strategy  was  the  ability  of 
certain  high-level  deliberative  behaviors  to  influence  or  even  inhibit  the  intermediate  and  low-level 
reactive  behaviors,  such  as  disabling  collision  avoidance  just  prior  to  docking  with  the  recharging 
station.  All  on-board  processing  was  performed  on  a  1-mHz  Synertek  SYM-1  single-board  computer 
with  just  36  kilobytes  of  RAM,  programmed  in  6502  assembly  language. 

Table  2.  The  layered  behavior  structure  employed  on  ROBART I  for  autonomous  navigation  and 
collision  avoidance  allowed  certain  high-level  deliberative  behaviors  to  influence  or  even  disable 

intermediate  and  low-level  reactive  behaviors. 


Level 

Behavior 

Resulting  Action 

High 

Radar 

Survey 

Dock 

Look  ahead  for  potential  obstacles 

Look  for  opening  in  forward  hemisphere 
Home  on  recharging-station  beacon 

Intermediate 

Wander 

Wall  Hugging 

Seek  clear  path  along  new  heading 

Follow  adjacent  wall  in  close  proximity 

Low 

Proximity  Reaction 
Impact  Reaction 

Veer  away  from  close  proximity 

Veer  away  from  physical  contact 

In  support  of  the  high-level  Radar  and  Survey  behaviors,  a  custom-designed  near-infrared 
proximity  sensor  mounted  on  the  head  provided  reliable  detection  of  diffuse  wall  surfaces  for  ranges 
out  to  about  6  feet  (Figure  78).  Lateral  resolution  was  sufficient  to  reliably  locate  the  edge  of  an  open 
doorway  to  within  1  inch  of  arc  at  5  feet  (Everett,  1982).  No  distance-measurement  capability  was 
provided,  however,  other  than  any  detected  target  was  somewhere  within  the  effective  range  of 
approximately  6  feet.  This  sensor,  which  could  be  reoriented  up  to  100  degrees  on  either  side  of 
centerline  by  panning  the  head,  was  extremely  useful  in  locating  open  doors  and  clear  zones  for 
travel. 


Figure  78.  ROBART  I  was  generously  equipped  with  a  variety  of  proximity  sensors,  feeler  probes, 
tactile  bumpers,  and  an  LM-1812-baseti  sonar  for  obstacle  detection  and  avoidance  (Everett,  2005). 
The  parabolic  reflector  at  top  was  part  of  a  custom  near-infrared  scanner  used  to  detect  open 
doorways  and  unobstructed  areas  for  forward  transit. 
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The  navigational  behaviors  outlined  in  Table  2  collectively  supported  the  robot’s  hallway- 
navigation  scheme  (Davis,  2011),  aided  by  the  recharger  beacon  being  suitably  positioned  to  assist 
the  robot  in  finding  the  hallway  (Figure  79).  Once  in  the  hall,  the  robot  would  move  parallel  to  the 
walls  in  a  reflexive  fashion,  guided  by  10  short-range,  near- infrared  proximity  sensors  on  the  front 
and  sides  (see  Figure  78),  plus  the  previously  discussed  head-mounted  proximity  scanner.  General 
orientation  in  the  hallway  could  be  determined  from  which  direction  afforded  a  view  of  the  beacon. 
With  an  a  priori  linked-list  representation  of  where  the  rooms  were  situated  relative  to  this  hallway, 
the  robot  could  proceed  to  any  given  room  by  simply  counting  off  the  correct  number  of  open 
doorways  on  the  appropriate  side  (Everett,  1996). 


Figure  79.  General  floor  plan  (recalled  from  memory)  of  the  Everett  residence  on  Rickett’ s  Road  in 
Monterey,  CA,  showing  the  recharging  station  (small  circle)  at  the  right-most  end  of  the  hallway 
(Everett,  2005).  The  beacon  on  the  recharger  could  be  remotely  activated  by  the  robot  to  serve  as  a 
navigational  aid. 

The  side-looking  proximity  sensors  (see  Figure  78),  with  maximum  range  set  to  about  16  inches, 
kept  the  wandering  robot  reasonably  centered  while  transiting  the  3 -foot- wide  hallway.  The  room- 
entry  behavior  was  tuned  by  trial-and  error  adjustment  of  head  scan  angle  0  for  doorway  detection 
(Figure  80).  Insufficient  head  deflection  would  cause  premature  detection,  resulting  in  the  robot 
turning  too  soon,  while  too  much  deflection  (0i)  caused  the  robot  to  turn  too  late.  The  forward- 
looking  proximity  sensors  enabled  last  minute  heading  adjustments  as  needed  to  ensure  collision- free 
doorway  penetration.  All  proximity  sensors  prevented  the  robot  from  entering  congested  spaces,  as 
for  example  through  the  open  doorway  of  a  small  closet. 


Figure  80.  Turn  initiation  was  tuned  by  varying  head  scan  angle  9.  For  02,  the  head-mounted 
proximity  sensor  has  just  detected  the  door  opening  at  wall  position  6,  which  will  result  in  the  optimal 
arc  shown  by  the  dashed  line. 
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Intended  as  a  simplistic  demonstration  of  technical  feasibility,  ROB  ART  I  was  built  on  a  limited 
budget,  funded  by  Lieutenant  Commander  Everett  with  approximately  $8000  of  his  own  money.  This 
philosophy  assumed  that  if  the  security-robot  concept  could  be  successfully  implemented  under  such 
primitive  development  strategies,  a  reasonable  extrapolation  would  show  significant  promise  for  a 
more  sophisticated  second-generation  version.  The  success  of  this  approach  started  a  media  frenzy 
that  led  to  Everett’s  assignment  in  November  1982  as  special  assistant  for  Robotics  to  Vice  Admiral 
Earl  Fowler,  Commander,  Naval  Sea  Systems  Command,  in  Washington,  DC  (Appendix  C) 

(Newton,  2006;  Davis,  201 1).  The  Naval  Postgraduate  School’s  Public  Affairs  Officer  relocated  for 
a  week  to  the  Everett  household  to  answer  the  phone,  which  rang  again  almost  as  soon  as  it  was  hung 
up  (Davis,  2011). 


Figure  81 .  Lieutenant  Commander  Everett  explains  the  perception  sensors  supporting  ROBART I 
to  a  television  reporter  at  the  new  Robotics  Lab,  NSWC  White  Oak.  The  robot  played  a  prominent 
role  in  numerous  demonstrations  at  this  facility  from  1983  through  1985,  when  it  was  showcased  at 
Expo  ’86  in  Vancouver,  BC. 

Early  the  following  year,  ROBART  I  was  loaned  to  the  Naval  Surface  Weapons  Center  (NSWC), 
White  Oak,  MD  (Figure  81),  entrusted  to  the  watchful  care  of  MIT  AI  Lab  co-op  student  Anita  Flynn 
(Figure  82),  a  later  pioneer  in  the  field  of  micro-robotics.  Towards  the  end  of  1985,  ROBART  I  was 
shipped  from  NSWC  to  Vancouver,  BC,  for  yearlong  exhibition  in  the  Design  2000  pavilion  at  the 
EXPO  '86  world’s  fair  (Davis,  2011).  The  well-traveled  feasibility  prototype  was  returned  to  Everett 
at  NOSC  in  1987,  where  it  ultimately  became  a  static  display  alongside  its  relatives  in  the  museum 
area  of  Building  624,  Seaside. 
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a) 


b) 


Figure  82.  a)  Co-op  student  Anita  Flynn  with  two  of  her  robots  at  the  MIT  Al  Lab.  b)  Ms.  Flynn  took 
charge  of  ROBART I  at  the  NSWC  Robotics  Lab,  White  Oak,  MD  (Davis,  201 1 ),  and  often  worked 
nights  and  weekends  in  Everett’s  basement  assisting  with  software  development  and  testing  of 
ROBART  II  (see  later  section). 

Ground  Surveillance  Robot  (1981-1 987) 

Funded  by  the  USMC,  the  previously  mentioned  Ground  Surveillance  Robot  (GSR)  project 
explored  the  development  of  a  modular  and  flexible  distributed  architecture  for  the  integration  and 
control  of  complex  robotic  systems  (Harmon,  1982;  Aviles,  Gage,  Harmon,  &  Bianchini,  1985).  The 
7-ton  Vietnam- War-era  M-114  Command  and  Reconnaissance  Carrier  donated  by  the  Army 
Research  Laboratory  (ARL)  was  outfitted  by  the  Unmanned  Systems  Branch  (Code  442)  to  serve  as 
a  fully  actuated  vehicle  testbed  (Figure  83).  The  long-term  goal  of  this  6.2  USMC  program  was  to 
explore  a  number  of  identified  issues  for  military  UGV  applications,  such  as  reliability,  security,  and 
multi-robot  coordination  (Harmon  &  Gage,  1984). 


Figure  83.  a)  The  GSR  was  equipped  with  an  array  of  fixed  and  steerable  ultrasonic  sensors  for 
following  a  lead  vehicle  or  pedestrian,  plus  an  agricultural  John  Deere  Doppler  ground-speed 
sensor  (photo  center),  b)  The  GSR  interior  offered  ample  space  for  a  network  of  IBM  Personal 
Computers,  circa  1985. 
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From  an  operational  perspective,  the  GSR  was  intended  to  transit  from  a  known  geographic 
location  to  a  designated  destination  over  unknown  natural  terrain  (Harmon,  1986): 

“Although  the  GSR  must  develop  a  map  of  the  terrain  from  its  starting 
point  to  its  goal  and  plan  its  route  from  that  information  alone,  it  must 
also  improve  its  performance  by  using  a  computerized  terrain  map  of  the 
appropriate  territory  if  available.” 

The  GSR  also  had  to  avoid  obstacles  while  in  transit,  as  further  explained  by  Harmon  (1987a): 

“Obstacle  avoidance  capability  has  been  implemented  by  fusing 
information  from  vision  and  acoustic  ranging  sensors  into  local  goals 
and  avoidance  points.  The  influence  of  these  points  is  combined  through 
potential  field  techniques  to  accomplish  obstacle  avoidance  control. 

Distant  terrain  characteristics  are  identified  using  information  from  a 
gray-level  vision  system,  a  color  vision  system,  and  a  computer- 
controlled  laser  ranging  sensor.” 

The  laser  rangefinder  is  shown  in  Figure  84a,  with  a  close-up  of  the  video  sensors  in  Figure  84b. 


Figure  84.  a)  A  key  component  of  GSR  development  was  an  upward  facing  laser  rangefinder 
mounted  beneath  a  3-DOF  (degree-of-freedom)  articulated  mirror  for  redirecting  the  beam,  and 
a  lead-screw  lift  mechanism  for  elevation,  b)  The  color  and  monochrome  vision  cameras  were 
mounted  adjacent  to  the  mirror  assembly.  Note  office  trailer  in  background  left. 

The  vehicle  perception,  planning,  and  control  subsystems  were  coordinated  via  a  distributed- 
blackboard  architecture  implemented  on  multiple  PCs  as  shown  in  Figure  85  (Harmon  et  al.,  1986). 
Terrain  features  identified  by  the  video  sensors  and  reasonably  located  by  the  laser  were  to  be  used 
by  a  planning  engine  to  determine  an  appropriate  path  to  some  visible  waypoint  in  the  direction  of 
the  final  goal  (Harmon,  1987a).  This  intuitive  process,  which  closely  imitated  how  a  human  might 
navigate  cross  country,  was  to  be  repeated  in  iterative  fashion  to  generate  a  succession  of  vehicle 
movements  from  waypoint  to  waypoint  to  reach  the  intended  destination. 
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Figure  85.  Blackboard  model  of  subsystem  interaction  (adapted  from  Harmon,  1986).  The  camera 
and  laser  navigation  sensors,  for  example,  had  to  interact  in  building  a  terrain  map  by  using  the 
camera-generated  contrast  map  to  guide  discrete  spatial  sampling  performed  by  the  laser 
rangefinder  (Harmon,  1987b). 

Since  this  was  one  of  the  first  pioneering  attempts  at  autonomous  navigation  in  exterior 
environments,  there  understandably  were  many  practical  problems  hampering  successful 
implementation  at  the  time.  Key  deficiencies  were  adequate  sensor  subsystems  that  could  effectively 
perceive  the  environment,  as  well  as  more  accurate  vehicle  localization  subsystems  that  could  geo¬ 
reference  the  acquired  perception  data  in  absolute  coordinates.  As  summarized  by  Harmon  (1987a): 

“The  experience  from  implementing  this  autonomous  vehicle  has  indicated 
the  need  for  an  integrated  set  of  debugging  tools  which  make  the  faults 
in  subsystem  hardware  and  software  more  distinguishable.” 

Using  monochromatic  vision  and  both  fixed  and  steerable  ultrasonic  sensors,  the  GSR  did 
successfully  demonstrate  autonomous  following  of  both  a  lead  vehicle  and  a  walking  human  in  1986 
(Gage,  1995b).  Considerable  work  remained,  however,  to  achieve  point-to-point  autonomous 
navigation.  Overshadowed  by  DARPA’s  longer  term  Autonomous  Land  Vehicle  (ALV)  program 
(DARPA,  1986),  the  much  smaller  GSR  effort  was  terminated  the  following  year  by  the  USMC  to 
better  focus  limited  funding  on  nearer  term  teleoperated  solutions.  Development  status  at  this  time 
was  graphically  depicted  by  Harmon  (1987b)  in  Figure  86. 


Figure  86.  While  the  GSR  blackboard  (Figure  85)  was  tree-structured,  the  functional  level  was 
monolithic,  which  allowed  subsystems  to  interact  without  architecturally  imposed  limitations  (redrawn 
from  Harmon,  1987b).  The  Texture  Analysis,  Color  Vision,  Planning,  and  Planning  Interface 
subsystems  at  right  were  never  implemented. 
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Remote-Presence  Demonstration  System  (1 981-1 985) 

Intended  to  support  development  and  demonstration  of  remote-presence  technology,  the  Remote 
Presence  Demonstration  System  (nicknamed  “Green  Man”)  was  a  hydraulically  actuated 
anthropomorphic  master-slave  configuration  that  remotely  mimicked  the  upper  torso  of  a  human 
operator.26  The  early  prototype  shown  in  Figure  87a  was  assembled  in  1983  using  a  pair  of  MB 
Associates  arms  and  a  NOSC-developed  torso  and  head.27  The  1985  version  in  Figure  87b  provided 
additional  degrees  of  freedom  at  the  hip,  torso,  shoulder,  and  arms  (Umeda,  2015),  and  featured  an 
exoskeletal  master  controller  with  kinematic  equivalency  and  spatial  correspondence  of  the  torso, 
arms,  and  head. 


Figure  87.  a)  Early  “Green  Man”  prototype,  circa  1983.  b)  Developed  by  Code  53  at  the  NOSC 
Hawaii  Laboratory,  this  1985  version  (right)  mimics  the  head  and  torso  movements  of  operator  Dave 
Smith  (left).  The  stereo  cameras  on  this  configuration  were  integrated  by  avionics  contractor 
Teledyne  (Umeda,  2015). 

“Green  Man’s”  stereo-vision  system,  two  525-line,  35-degree  field-of-view  video  cameras  and  a 
corresponding  pair  of  video-camera  eyepiece  monitors  mounted  on  an  aviator's  helmet,  provided 
valuable  experience  in  telepresence  design.  Even  with  the  simplistic  claw  hands  and  no  force  or 
tactile  feedback,  novice  operators  could  readily  perform  manipulative  tasks  without  extensive 
training.  Continued  testing,  however,  clearly  showed  that  more  dexterous  manipulators,  force 
feedback,  and  a  high-resolution  vision  system  were  necessary  for  diver-equivalent  functionality.  The 
head-tracking/camera-pan-and-tilt  actuation  scheme,  on  the  other  hand,  was  successfully  transitioned 
to  the  Advanced  Teleoperator  Technology  “Dune  Buggy”  project,  discussed  in  a  later  section. 


26  The  name  reportedly  derives  from  the  original  color  of  the  hydraulic  fluid  that  was  visible  through  the  clear 
tubing,  which  was  at  some  later  point  was  changed  to  red. 

27  The  arm  was  built  in  the  1970s  by  Carl  Flattau  of  NOSC  Code  53 1  and  Herbert  L.  Mummery  of  M.B.  Associates 
(Rosheim,  1994). 
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Figure  88.  This  August  1988  photo  of  the  NOSC  “Green  Man”  posing  with  student  aid  JoJo  Aledo 
at  the  Hawaii  Lab  provides  a  close-up  view  of  the  wrist  actuation  scheme  for  the  right  arm. 

With  the  closure  of  the  NOSC  Hawaii  Laboratory  in  1992,  the  “Green  Man”  system  was  shipped 
to  NOSC  in  San  Diego  for  storage,  then  transitioned  the  following  year  to  Professor  Morris  Driels  at 
the  Naval  Postgraduate  School  (NPS)  in  Monterey,  CA  (Chatfield,  1995).  The  underlying  technology 
developed  during  the  course  of  the  Hawaii  project  was  instrumental  in  scoping  the  follow-on 
NAVSEA- funded  Teleoperator/Telepresence  System  (TOPS),  to  be  discussed  in  a  later  section 
(Rosheim,  1994). 

ROB  ART  //(1 982-1 992) 

Having  begun  construction  of  ROBART I  immediately  upon  arrival  at  the  Naval  Postgraduate 
School  in  1980,  Lieutenant  Commander  Everett  finished  his  thesis  at  NPS  several  months  early  and 
began  working  on  the  second-generation  ROBART II  in  mid- 1982  (Figure  90).  There  were  four 
general  objectives: 

1 .  Make  the  system  more  modular  to  facilitate  maintenance  and  upgrades 

2.  Employ  a  parallel-processing  hierarchy  of  distributed  microprocessors  (Figure  89) 

3.  Incorporate  a  more  sophisticated  mix  of  sensors  in  support  of  advanced  autonomy 

4.  Provide  a  more  finished  look  to  the  modular  body  structure 
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Figure  89.  Block  diagram  of  the  computer  architecture  on  ROBART II,  circa  1989.  The  number  of  on¬ 
board  microprocessors  was  ultimately  increased  to  13  (Davis,  2011). 

The  physical  structure  of  ROBART  II  consisted  of  a  cylindrical  upper  body  that  mated  with  a 
rectangular  mobility  base  (Figure  90a),  with  a  removable  electronics  cage  serviced  via  a  pair  of 
access  doors  (Figure  90b&c).  The  upper  housing  was  fashioned  from  a  30-inch  section  of  12-inch- 
diameter  plastic  pipe,  with  an  acrylic  cake  cover  supported  by  a  Lazy-Susan  bearing  that  formed  the 
head  pan  axis.  The  initial  mobility  base  was  a  functional  plywood  mockup  fitted  with  a  pair  of 
A-BEC  wheelchair  motors,  front  and  rear  castors,  and  a  lead-acid,  gel-cell  battery.  Once  the  mobility 
design  was  optimized,  this  temporary  plywood  base  was  replaced  by  a  hardened  aluminum  version 
with  a  black  fiberglass  shroud. 


Figure  90.  a)  The  initial  instantiation  of  ROBART  II  was  a  plastic-pipe  housing  on  top  of  a  functional 
plywood  prototype  of  the  mobility  base,  circa  June  1982.  b)  Rear  view  showing  computer  cage  and 
head  pan  assembly,  circa  August  1982.  c)  The  computer  module  could  be  easily  removed  for 
maintenance  (circa  1984). 
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Following  Everett’s  reassignment  to  NAVSEA  later  that  fall,  the  first  of  two  distinct  versions  of 
ROBART II began  to  evolve  in  his  basement  workshop  in  Springfield,  VA  (Figure  91a),  which 
consequently  came  to  be  known  as  the  “Virginia  version.”  This  second-generation  security  robot 
performed  essentially  the  same  functions  as  its  predecessor  ROBART  I,  but  with  a  multiprocessor 
architecture  that  enabled  parallel  real-time  operations  (Everett,  1985d).  Improved  performance  was 
further  addressed  through  significantly  increased  perception  and  more  precise  motion  control,  the 
latter  supported  by  phase-quadrature  optical  encoders  attached  to  the  drive-motor  armatures  as  shown 
in  Figure  91b  (Everett,  1985e). 


Figure  91 .  a)  This  early  version  of  ROBART  II  at  the  Naval  Surface  Weapons  Center  (NSWC),  White 
Oak,  MD,  was  upgraded  with  an  aluminum  second-generation  mobility  base,  circa  1983.  b)  Close-up 
view  of  the  drive  components,  showing  the  optical  shaft  encoders  retrofitted  to  a  pair  of  A-BEC 
wheel-chair  motors. 

As  shown  at  the  far  left  in  Figure  92,  the  collision-avoidance  sensor  suit  was  upgraded  from  that  of 
ROBART  I  with  the  addition  of  a  five-element  Polaroid  electrostatic  sonar  array,  and  a  sixth  sonar 
transducer  mounted  on  the  head  (Everett,  1985).  Note  the  partially  installed  tactile-bumper  strip 
around  the  bottom  of  the  mobility  base.  This  evolving  “Virginia  version”  of  ROBART  II  was  used  by 
Anita  Flynn  to  support  her  Master’s  thesis  at  MIT  (Flynn,  1985;  Everett  &  Flynn,  1986).  Ms.  Flynn 
worked  alternate  academic  quarters  during  her  undergraduate  career  in  the  NSWC  Autonomous 
Systems  Branch  (Davis,  2011),  then  headed  by  Ms.  Mary  E.  Lacey,  later  deputy  assistant  secretary  of 
the  Navy  for  Research,  Development,  Test  and  Evaluation. 
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Figure  92.  Russ  Werneth  of  NSWC  and  Lieutenant  Commander  Bart  Everett  of  SEA-90G  with 
ROBART  I,  ROBART II,  and  the  Odex  robot  by  Odetics  at  the  NSWC  Robotics  Lab,  circa  1985.  Note 
the  six  gold  Polaroid  electrostatic  sonar  transducers  on  ROBART  II,  versus  the  single  black 
piezoelectric  transducer  on  ROBART  I. 


One  of  the  more  impressive  autonomy  demonstrations  performed  by  ROBART  II  for  visitors  at  the 
NSWC  Robotics  Lab  was  a  robust  person- following  behavior.  The  sonar-transducer  layout  of  Figure 
92  helped  discriminate  a  near-field  human  target  from  other  reflective  surfaces  that  should  be 
avoided  (see  also  Figure  93a).  The  sensor-fusion  approach  exploited  the  vertical  aspect  ratio  of  an 
erect  human  body  in  conjunction  with  perceived  relative  motion  with  respect  to  the  robot.  Lateral 
displacement  of  the  human  target  detected  by  the  bottom  three  transducers  was  used  to  proportionally 
adjust  the  robot’s  heading.  This  algorithm  could  reliably  follow  a  moving  human  through  a  cluttered 
room  and  transit  a  narrow  28-inch  doorway. 


Figure  93.  a)  ROBART  II  at  NSWC.  Note  Banner  floor  scanner  for  negative  obstacles  on  the  front  of 
the  base,  and  the  two  tactile  bumper  strips  immediately  below.  U.S.  Patent  No.  4,596,412  was 
issued  to  H.R.  Everett  and  C.S.  Wright  for  this  bumper  design  on  24  June  1986.  b)  ROBART  II  upon 
arrival  in  San  Diego,  with  the  new  fiberglass  sonar-array  housing  in  place  just  below  the  head  for  a  fit 
check  prior  to  transducer  installation. 
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Following  Everett’s  transfer  to  NOSC  in  1986  (Figure  93b),  the  “California  version”  of  ROBART 
II  became  a  concept-development  surrogate  in  support  of  indoor  robot  autonomy,  with  initial  focus 
on  two  specific  technology  needs  (Everett  &  Bianchini,  1987).  The  first  of  these  addressed  the 
navigational  shortfalls  identified  by  SEA  90G  that  were  hindering  successful  implementation  of  a 
number  of  robotic  applications  requiring  mobility  (Gilbreath  &  Everett,  1988;  Babb,  1990).  To 
enable  successful  traversal  of  congested  surroundings,  the  robot  was  upgraded  with  numerous 
proximity  and  ranging  sensors  for  autonomous  mapping,  localization,  collision  avoidance,  and 
navigational  planning  (Figure  94). 


Figure  94.  a)  Rebecca  Everett  cleans  up  after  her  dad  during  the  upgrade  of  ROBART  IT s  lower 
sonar  array  in  their  San  Diego  kitchen,  b)  Interior  view  of  the  cylindrical  body  housing  with  the  head 
removed,  showing  the  two  12-channel  sonar  multiplexors  for  the  upper  sonar  array  shown  in  Figure 
93b. 

Code  535  electrical  engineer  Gary  Gilbreath  chose  a  cell-based  map  representation  for  the  robot’s 
world  model,  with  free  space  indicated  by  a  cell  value  of  zero  (a  non-zero  value  indicated 
occupancy).  This  approach  offered  the  following  advantages  (Everett  &  Gilbreath,  1989): 

•  The  operating  area  was  a  bounded  interior  space  where  a  relatively  coarse  grid  (i.e.,  3-inch 
resolution)  could  be  used  (Figure  95). 

•  The  traversability  of  a  square  could  be  statistically  represented  and  easily  changed. 

•  Objects  of  unknown  configuration  were  easily  added. 

•  A  simple  Lee  maze  router  could  be  used  for  path  planning  (Lee,  1961). 

•  Unique  coding  of  predefined  entities  (e.g.,  doorways,  recharging  station)  was  supported. 

This  early  implementation  of  “augmented  virtuality,”  wherein  the  battery-charger  location  and 
doorway  status  (open  or  closed)  were  reflected  in  the  world  model,  was  substantially  augmented  later 
under  ROBART  III  ( 1 992-2007). 
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Figure  95.  a)  Photo  of  Room  102  in  Building  F-36,  with  ROBART II  situated  at  Point  A  in  the 
adjacent  map  representation  of  Figure  95b.  b)  Resulting  straight-line  path  from  position  A  to  goal 
destination  B,  with  overlaid  circles  and  rectangles  representing  the  X-Y  locations  of  the  as  yet 
undetected  transient  objects  shown  in  Figure  95a. 

During  the  execution  of  this  path  segment,  the  collision-avoidance  sonar  array  detected  the  row  of 
cylinders  (Figure  95)  and  began  altering  the  associated  cell  probabilities  to  reflect  the  perceived 
obstructions.  When  the  robot  moved  to  within  the  collision  threshold  of  22  inches,  forward  motion 
was  halted  and  newly  mapped  objects  were  temporarily  grown  for  maneuvering  clearance.  This 
transient  growth  was  removed  after  the  new  path  segment  was  found  (Figure  96a).  The  newly  added 
dark-black  areas  represent  the  obstacles  detected  during  the  first  move.  Upon  executing  the  revised 
path,  the  robot  discovered  the  cart  and  planned  another  avoidance  maneuver  (Figure  96b). 


Figure  96.  a)  An  avoidance  maneuver  was  generated  by  the  path  planner  to  clear  the  row  of 
cylinders  shown  in  Figure  95a.  b)  Revised  path  from  robot’s  new  position  to  accommodate  the 
discovery  of  the  cart  shown  earlier  in  Figure  95a  (adapted  from  Everett,  Gilbreath,  &  Tran,  1990). 


The  improved  battery  charging  station  shown  in  Figure  97,  compatible  with  the  entire  ROBART 
series,  had  been  constructed  in  Virginia  by  Everett  just  prior  to  his  1986  transfer  to  the  NOSC  in  San 
Diego,  CA.  As  with  ROBART  I,  the  homing  beacon  was  activated  by  a  garage-door  RF  link, 
whereupon  a  current-limited  sense  voltage  was  applied  to  the  recharging  contacts  so  a  valid 
connection  could  be  perceived  by  the  robot  upon  docking.  The  charging  station  also  detected  this 
connection  and  activated  the  battery-charger  power  supply  after  the  mating  contacts  had  debounced 
(Everett,  Gilbreath,  &  Tran,  1990).  U.S.  Patent  No.  5,045,769  for  an  “Intelligent  Battery  Charging 
System”  was  awarded  H.  R.  Everett  on  3  September  1991. 
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Figure  97.  a)  ONR  summer  intern  Lisa  Dakis  with  the  new  charger,  compatible  with  the  entire 
ROBART  series,  b)  ROBART  ll’s  head  mounted  sonar  provided  continuous  range-to-beacon 
measurements  during  approach,  while  the  collision-avoidance  sonar  array  just  above  the  mobility 
base  detected  obstacles  to  be  added  to  the  world  model. 


The  second  thrust  was  aimed  at  producing  a  robust  automated  security  system  exhibiting  a  high 
probability  of  detection,  with  the  equally  important  ability  to  distinguish  between  actual  and  nuisance 
alarms  (Everett,  Gilbreath,  &  Alderson,  1988).  ROBART II  was  already  equipped  with  a  multitude  of 
environmental  sensors  that  monitored  system  and  room  temperature,  relative  humidity,  barometric 
pressure,  ambient  light,  noise  levels,  toxic  gas,  smoke,  and  fire.  Intrusion  detection  was  addressed 
through  the  use  of  infrared,  optical,  ultrasonic,  microwave,  and  video  motion  detection,  as  well  as 
vibration  monitoring  and  discriminatory  hearing  (Figure  98). 
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Figure  98.  ROBART II  was  equipped  with  an  array  of  environmental,  intrusion  detection,  and 
navigation  sensors.  The  upgraded  Polaroid  sonar  system  featured  36  electrostatic  transducers 
(versus  the  original  six)  in  two  separate  arrays.  The  upper  body  easily  detached  from  the  mobility 
base  upon  removal  of  four  quick-release  pins. 

To  increase  the  probability  of  detection  and  reduce  nuisance  alarms,  two  new  sensor  modalities 
were  added  to  the  Intelligent  Security  Assessment  System  (Everett,  Gilbreath,  &  Tran,  1990).  A  line- 
based  video-motion-detection  scheme  allowed  a  6502-based  single-board  computer  to  digitize  any 
three  horizontal  lines  of  a  composite  video  image  (Figure  99).  Developed  by  electrical  engineer 
Theresa  Tran,  the  software  would  monitor  each  of  these  lines  for  changes  indicative  of  motion, 
reconfigure  line  selection  to  focus  on  suspected  anomalies,  then  compare  the  perceived  aspect  ratio 
of  the  disturbance  to  a  human  target  (Everett,  Gilbreath,  &  Tran,  1990).  US  Patent  No.  5,034,817  for 
a  “Reconfigurable  Video  Line  Digitizer”  was  issued  to  H.R.  Everett  on  23  July  1991. 28 


28  Originally  developed  for  ROBART II  in  1989,  production  versions  of  this  system  were  installed  on  ROBART  III 
and  the  MDARS-Interior  robot,  both  discussed  in  later  sections. 
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Figure  99.  Block  diagram  of  the  8-bit  6502-based  “Reconfigurable  Video  Line  Digitizer  (VLD)”  built 
by  R.J.  Brachman  Associates,  circa  January  1990.  This  new  sensor  modality  was  activated  in 
response  to  primary  alerts  to  better  discriminate  between  actual  and  nuisance  alarms  (Everett, 
Gilbreath,  &Tran,  1990). 

The  second  sensor  upgrade  was  a  directional  head-mounted  acoustic  array  that  determined  the 
relative  bearing  to  the  source  of  impulse  noise  such  as  breaking  glass  or  a  dropped  object  (Figure 
100).  The  omnidirectional  microphones  and  preamplifier  stages  were  extracted  from  three  Radio 
Shack  baby  monitors  and  coupled  to  operational-amplifier  circuits  configured  as  threshold 
comparators,  the  outputs  of  which  were  monitored  by  the  software.  The  known  array  geometry  and 
the  time  of  detection  for  each  microphone  element  would  yield  the  direction  to  the  source  to  within  a 
few  degrees,  adding  yet  another  data  set  for  both  temporal  and  spatial  correlation  (Everett,  Gilbreath, 
&  Tran,  1990).  The  6502  software  for  this  acoustic  array  was  also  written  by  Theresa  Tran. 


Figure  100.  a)  From  left  to  right,  Commander  Bart  Everett  and  Gary  Gilbreath  meet  with  John 
Holland  of  Cybermotion  and  Tim  Papayianis  of  the  U.S.  Army  Armament  Research  Development 
and  Engineering  Center  regarding  ROBART II  technology  of  interest  to  the  Army’s  Mobile  Detection 
Assessment  Response  System  (MDARS).  b)  Closer  view  of  the  head-mounted  acoustic  array  for 
intruder  detection  (see  also  Figure  106a). 
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In  addition  to  the  basic  smoke  and  gas  sensors,  seven  different  kinds  of  intrusion-detection 
modalities  were  ultimately  employed  on  ROBART II,  to  include  passive  infrared,  microwave,  optical, 
vibration,  acoustical,  sonar,  and  video.  Time-stamped  sensor  status  as  well  as  environmental 
conditions  were  displayed  as  shown  in  Figure  101,  and  could  be  overlaid  on  live  video  from  the 
robot’s  camera  (Smurlo  &  Everett,  1993).  Written  by  Gary  Gilbreath,  all  high-level  planning  and 
security-assessment  software  ran  on  a  desktop  PC/AT  computer,  connected  to  the  remote  vehicle  via 
a  1200-baud  Repco  RF  modem  (Everett,  Gilbreath,  and  Tran,  1990). 
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Figure  101 .  In  the  upper  left-hand  window  of  the  Security  Display,  sensor  modalities  not  currently 
active  were  depicted  in  reverse  video  (gray  background).  Just  to  the  right,  individual  sensors  within 
the  active  groups  were  portrayed  in  reverse  video  (light  gray  background)  when  alarmed  (adapted 
from  Everett,  Gilbreath,  and  Tran,  1990). 

The  Intelligent  Security  Assessment  System  achieved  a  high  probability  of  detection  through 
fusion  of  a  variety  of  motion-detection  sensor  outputs  (Figure  102),  while  simultaneously  reducing 
the  nuisance-alarm  rate  through  cross-correlation  (Everett,  Gilbreath,  and  Tran,  1990).  The 
importance  of  this  latter  step  became  apparent  in  earlier  work  with  ROBART  I  (Everett,  1980). 
Traditional  fixed-installation  security  sensors  can  be  optimally  mounted  within  a  building  to 
minimize  the  chances  of  spurious  interference,  which  is  obviously  not  possible  when  installed  upon  a 
robotic  platform  that  roams  about.  U.S.  Patent  No.  4,857,912  for  an  “Intelligent  Security  Assessment 
System”  was  awarded  to  H.R.  Everett  and  Gary  Gilbreath  for  this  concept  on  15  August  1989. 


Figure  102.  Block  diagram  of  the  Intelligent  Security  Assessment  System  on  ROBART  II  (adapted 
from  Everett,  Gilbreath,  and  Tran,  1990).  This  concept  was  later  expanded  to  allow  sensor  fusion 
with  fixed-installation  motion  sensors  inside  the  facility  patrolled  by  the  robot  (Figure  103). 
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The  assessment  algorithm  was  upgraded  ini  990  to  include  consideration  of  any  fixed- installation 
motion  sensors  (Figure  103),  thereby  allowing  a  mobile  robot  to  operate  in  a  secure  area  already 
protected  by  stand-alone  motion  sensors.  U.S.  Patent  No.  5,202,661  for  a  “Method  and  System  for 
Fusing  Data  from  Fixed  and  Mobile  Security  Sensors”  was  subsequently  awarded  to  H.R.  Everett 
and  Gary  Gilbreath  on  13  April  1993.  A  final  enhancement  involved  integrating  historical  sensor  data 
to  assess  temporal  changes  (Smurlo  &  Everett,  1994),  resulting  in  U.S.  Patent  No.  5,493,273  for  a 
“System  for  Detecting  Perturbations  in  an  Environment  Using  Temporal  Sensor  Data,”  awarded  to 
H.R.  Everett  and  R.P.  Smurlo  on  20  February  1996. 


Navigational  Layer 


Figure  103.  ROBART ll’s  expanded  world  model  employed  two  additional  bit-map  layers  to 
represent  the  coverage  areas  of  both  fixed  and  mobile  intrusion-detection  sensors  (adapted  from 
Everett,  Gilbreath,  and  Tran,  1990). 

Referring  now  to  Figure  104,  a  detected  intruder  reported  by  one  of  the  fixed- installation  motion 
sensors  would  be  relayed  to  the  host  computer,  which  alerted  the  guard  by  a  beep  from  the  console. 
The  Status  and  Environmental  windows  shown  earlier  in  Figure  101  were  then  replaced  by  the 
current  floorplan  map.  The  security-assessment  software  determined  that  the  alarmed  sensor  was  not 
triggered  by  the  moving  robot,  since  its  current  X-Y  position  was  not  within  the  designated  sensor 
coverage  area.  The  Planner  consequently  rerouted  the  robot  to  a  location  where  its  on-board  sensors 
could  observe  the  area  of  disturbance  and  locate  the  intruder.  If  after  a  designated  period  of  time  with 
no  confirmation  from  the  robot’s  sensors,  the  assessment  software  would  downgrade  the  threat  to  a 
nuisance  alarm. 
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Figure  104.  The  respective  coverage  areas  for  two  fixed-installation  motion  sensors  on  the  bottom 
wall  are  overlaid  on  the  X-Y  floorplan  map.  Highlighted  here  by  the  diagonal  arrow,  the  calculated 
position  of  a  confirmed  intruder  is  graphically  depicted  near  the  doorway  to  the  adjoining  room 
(adapted  from  Everett,  Gilbreath,  and  Tran,  1990). 
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In  1989,  a  reflexive-teleoperated  driving  mode  (now  commonly  known  as  “guarded  motion”)  was 
added  to  ROB  ART  II  to  test  mobility  behaviors  that  could  reduce  the  driving  burden  imposed  upon 
military  operators  of  more  simplistic  man-portable  UGVs  (Laird  &  Everett,  1990).  The  collision- 
avoidance  sensors,  originally  intended  to  provide  an  envelope  of  protection  during  autonomous 
transit,  were  called  into  play  during  teleoperation  to  minimize  the  possibility  of  operator  error  (Figure 
105).  Although  ROB  ART  II  was  never  intended  to  be  remotely  driven  by  a  human  operator,  reflexive 
teleoperation  was  one  of  the  first  UGV  behaviors  for  which  the  system  served  as  a  software- 
developmental  surrogate.  The  commanded  speed  and  direction  of  the  mobility  base  were  servo- 
controlled  in  response  to  local  sensor  inputs  to  keep  the  robot  from  running  into  obstructions  (Laird 
&  Everett,  1990). 


Figure  105.  Center,  left,  and  right  zones  of  coverage  for  the  proximity  (shaded)  and  ultrasonic 
(concentric  arcs)  sensors.  The  rate  of  turn  was  proportional  to  the  degree  of  obstacle  encroachment 
into  the  intended  path  of  travel.  US  Patent  No.  5,307,271  was  awarded  H.R.  Everett  and  Gary 
Gilbreath  for  this  feature  on  26  April  1994. 


Also  on  the  agenda  was  the  pursuit  of  localization  techniques  to  better  support  autonomous 
navigation  in  indoor  environments  (Everett  &  Gage,  1995b).  In  the  late  1980s  to  early  1990s,  before 
the  ready  availability  of  scanning  lasers  and  simultaneous-localization-and-mapping  (SLAM) 
algorithms,  many  different  approaches  were  implemented  for  subsequent  evaluation  (Everett, 
Everett,  Gilbreath,  &  Tran,  1990;  Borenstein,  Everett,  Feng,  &  Wehi,  1997): 


Ultrasonic  transponder  triangulation 
Fluxgate  compass 
Rate  gyro 

Polarized  optical  heading  reference 
Video  image  processing 
Ultrasonic  signature  matching 
Ultrasonic  wall  referencing 
Tactile  wall  referencing 


Real-time  wall  following 
Guidepath  following 
Beacon  following 
Beacon  referencing 
Doorway  transit  referencing 
Lateral  retroreflective  sensing 
Overhead  retroreflective  sensing 
RF  referencing 


The  doorway- transit  referencing  system  consisted  of  two  Banner  Engineering  near-infrared  SM31 
emitter/detector  pairs  mounted  on  either  side  of  the  head  as  shown  in  Figure  106.  These  units  were 
angled  slightly  so  their  optical  axes  crossed  at  the  height  of  a  standard  doorway,  enabling  a  diffuse 
surface  at  this  elevation  only  to  reflect  emitted  energy  back  to  the  corresponding  detectors.  Careful 
comparison  of  the  times  of  overhead  detection  for  each  sensor  pair  enabled  calculation  of  the  robot’s 
heading  relative  to  the  known  doorway  orientation,  along  with  its  longitudinal  position  along  the  path 
of  travel.  The  lateral  position  of  the  robot  relative  to  the  door  frame  was  simultaneously  determined 
by  reading  the  left-  and  right-most  range  values  measured  by  the  upper  Navigational  Ultrasonic 
Array  (Everett,  Everett,  Gilbreath,  &  Tran,  1990). 
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Figure  106.  a)  The  two  yellow  Banner  emitter/detector  pairs  atop  the  head  were  angled  slightly  so 
their  respective  optical  axes  crossed  at  the  height  of  a  standard  doorway,  b)  ROBART II  prepares  to 
exit  Room  1 15  of  Building  622  Seaside  via  an  open  doorway,  which  would  automatically  provide  an 
X-Y-0  localization  update. 


Quite  a  few  reports,  conference  papers,  and  magazine  articles  were  published  on  the  above  work, 
resulting  in  engineers  from  the  US  Armament  Research,  Development,  and  Engineering  Center 
(ARDEC)  at  Picatinny  Arsenal  contacting  NOSC  in  1989  for  technical  support  (Babb,  1990;  Newton, 
2006).  The  resulting  technology  transfer  to  the  Army’s  MDARS-Interior  program  of  record  (Figure 
107a)  is  discussed  later  in  the  Project  Summaries  section.  With  the  advent  of  ROBART  III  in  1992 
(Figure  107b),  the  role  of  ROBART  II  as  a  concept-development  surrogate  came  to  an  end.  In  spite  of 
having  been  built  at  home  from  hobbyist-grade  components,  this  second-generation  security  robot 
proved  to  be  an  amazingly  reliable  piece  of  equipment,  with  only  four  documented  cases  of  hardware 
failure  since  officially  coming  to  life  in  early  1983. 


Figure  107.  a)  The  MDARS-Interior  prototype  on  patrol  in  Building  F-36,  Seaside,  circa  1994. 
b)  Early  instantiation  of  ROBART  III  in  Building  622,  Seaside,  circa  1994. 
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This  mean  time  between  failure  is  rather  noteworthy,  considering  the  workout  given  the  system 
during  its  20-year  lifetime  -  records  indicate  ROBART //performed  in  53  live  demonstrations  for 
visiting  students,  faculty,  scientists,  and  government  officials  in  1987  alone.  The  robot  continued  to 
provide  demonstrations  for  visitors,  however,  and  remained  on  line  without  a  power  interruption 
from  1988  to  2002,  when  a  support  contractor  disconnected  its  recharging  station  over  a  weekend, 
allowing  the  on-board  battery  to  go  dead.  Dr.  John  Silva  (Code  014)  of  NOSC  estimated  the 
availability  of  ROBART  //probably  saved  the  Navy  a  million  dollars  in  development  costs  (Davis, 
2011)  while  facilitating  significant  achievements  in  path  planning,  collision  avoidance,  localization, 
sensor  fusion,  and  command  and  control. 

Advanced  Teleoperator  Technology^ 983-1 985) 

Managed  by  Dave  Smith  (Code  53 1)  of  the  Hawaii  Lab,  the  USMC  Advanced  Teleoperator 
Technology  project  controlled  Chenoweth  dune  buggies  with  helmet-mounted-display  technology 
developed  under  the  Remote-Presence  Demonstration  System  project  in  1981  (Spain,  1987).  The 
orientation  of  the  operator’s  head  would  be  detected  and  mimicked  at  the  UGV  (Figure  108),  so  he  or 
she  saw  the  3D  scene  from  the  vehicle  driver's  seat  perspective.  The  command-and-control  station 
was  laid  out  just  like  the  remote  vehicle,  with  steering  wheel,  brakes,  gas  pedal,  shift  knobs,  etc.,  all 
in  their  proper  places.  The  idea  was  that  any  Marine  who  could  drive  a  dune  buggy  could  remotely 
drive  the  UGV  with  minimal  training. 


Figure  108.  A  stereo-camera  pan-and-tilt  mechanism  situated  in  the  driver’s  seat  mimicked  the 
operator’s  head  movements  in  the  control  van,  as  sensed  by  the  Polhemus  3D  tracking  system. 

After  a  few  minutes  of  remote  driving,  USMC  operators  began  to  feel  like  they  were  actually  sitting  in 
the  vehicle  itself. 

Referring  now  to  Figure  109,  the  Polhemus  3D  tracking  system  at  right  calculated  the  spatial 
orientation  of  the  small  blue  transducer  shown  on  top  of  the  operator’s  helmet  relative  to  a  stationary 
reference  directly  above  (not  shown).  This  3D-pose  information  was  then  used  to  command  the  slave 
pan-and-tilt  unit  at  left  to  assume  an  identical  pose.  This  approach  allowed  the  operator  to  intuitively 
redirect  the  robot’s  gaze  in  both  azimuth  and  pitch,  simply  by  moving  his  or  her  head.  System  lag 
introduced  by  the  remote  pan-and-tilt  actuation  scheme  and  video-signal  propagation  delays  caused 
decoupling  of  the  human  vestibular-ocular  reflex,  however,  causing  some  operators  to  experience 
symptoms  of  vertigo. 
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Figure  109.  The  Polhemus  3D  tracking  system  calculated  the  spatial  orientation  of  the  small  blue 
transducer  (upper  right)  on  top  of  the  operator’s  helmet  relative  to  a  stationary  reference  directly 
above  (not  shown).  This  3D-pose  information  was  used  to  command  the  slave  pan-and-tilt  unit  at  left 
to  assume  an  identical  posture. 

The  prototype  weapon  payload  for  the  dune  buggy  consisted  of  an  M60  machine  gun  mounted  on  a 
custom-built  limited-motion  pan-and-tilt  unit  attached  to  the  roll  bar  as  shown  in  Figure  1 10  (Umeda, 
2015).  Live-fire  tests  were  conducted  at  the  Ulupau  Crater  firing  range  at  Marine  Corps  Air  Station 
Kaneohe  (now  Marine  Corps  Base  Hawaii),  followed  by  a  more  extensive  evaluation  at  Fort  Lewis, 
WA.  To  assess  remote-presence  driving  effectiveness  under  degraded  visibility  conditions,  both 
daytime  and  nighttime  (less  than  %  moon)  runs  were  conducted  over  a  predefined  course  at  speeds  up 
to  30  miles  per  hour.  Camera  sensitivity  was  enhanced  with  Gen  2+  night- vision  systems,  resulting 
in  low-resolution  monochromatic  video,  yet  drivers  were  able  to  successfully  negotiate  the  course 
using  visual  depth  cues  provided  by  stereo. 


Figure  110.  Live-fire  exercises  demonstrated  the  ability  to  position  the  teleoperated  dune  buggy  at  a 
2-  to  4-kilometer  standoff  distance  with  remote  pan,  tilt,  and  trigger  control  of  the  weapon  system.  To 
facilitate  aiming,  an  additional  video  camera  was  attached  to  the  M60  gunsight. 
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Airborne  Remotely  Operated  Device  (1 984-1 988) 

The  Airborne  Remotely  Operated  Device  (AROD)  was  a  small  ducted-fan  vertical-take-off-and- 
landing  (VTOL)  air  vehicle  that  could  easily  translate  through  the  air  and  provide  short-range  aerial 
surveillance.  The  project  was  initiated  at  the  NOSC  Hawaii  Laboratory  in  1984  as  part  of  the  US 
Marine  Corps  Exploratory  Development  (6.2)  Surveillance  Program.  This  effort  was  continued  as 
part  of  the  Ground  Air  Telerobotic  Systems  (GATERS)  Advanced  Technology  Demonstration  (6.3A) 
program  through  late  1980,  along  with  the  ground-based  Teloperated  Vehicle  (TOV).  Developed  by 
Moller  International  as  a  subcontractor  to  Perceptronics,29  the  first-generation  AROD  prototype  was 
electrically  powered  via  a  high-voltage  tether  from  the  ground  station,  and  was  small  enough  to  be 
carried  by  a  single  person. 


Figure  111.  The  early  NOSC  AROD  prototype  was  an  electrically  powered  tethered  quadrotor  built 
by  Moller  International,  Davis,  CA. 

While  appropriate  for  concept  demonstration,  the  electrical  tether  was  deemed  impractical  for 
military  operations.  Developed  for  NOSC  by  Sandia  National  Laboratory,  the  second-generation  air 
vehicles  were  powered  by  a  26-horsepower  two-stroke  gasoline  engine  driving  a  single  lifting 
propeller  (Figure  1 12).  Servo-driven  vanes  located  at  the  bottom  of  the  air  duct  controlled  vehicle 
attitude,  allowing  hover,  multi-directional  translation,  and  rotation  about  the  vertical  axis.  An 
automatic  flight  control  system  helped  maintain  vehicle  stability.  The  fiber-optic  cable  provided 
bidirectional  communications  with  a  small  ground  control  unit,  using  a  radio  link  as  backup.  The  5- 
kilometer  spool  of  optical  fiber  carried  aboard  AROD  could  support  a  2-kilometer  round  trip  or  a  5- 
kilometer  one-way  mission. 


29  Moller  International  was  previously  known  as  Discojet  Corporation,  incorporated  in  1971. 
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Figure  112.  a)  Conceptual  drawing  of  the  second-generation  AROD  vehicle  (redrawn  from  Bassett, 
1987).  b)  As  built  for  NOSC  by  Sandia  National  Laboratories,  this  ducted-fan  AROD  prototype  was 
gasoline  powered  (see  also  Figure  113a). 

A  three-degree-of-freedom  joystick  on  the  operator  control  unit  controlled  pitch,  roll,  and  yaw, 
while  a  thumbwheel  on  the  stick  controlled  altitude  (Figure  113b).  As  with  the  earlier  dune-buggy 
efforts  under  the  ATT  project,  a  helmet-mounted  display  provided  stereo  vision  to  the  operator,  who 
could  aim  the  on-board  camera  pair  using  intuitive  head  movements.  Although  the  vehicle  was 
successfully  tested  in  free  flight,  control  instabilities  prevented  it  from  realizing  its  full  range  of 
performance.  AROD  development  was  consequently  discontinued  when  funding  limitations  caused 
the  Ground  Air  TeleRobotic  System  (GATERS)  program  (see  next  section)  to  shift  focus  to  the 
TeleOperated  Vehicle  project.  This  concept  was  revisited  later  under  the  Air  Mobile  Ground  Security 
and  Surveillance  System  (AMGSS),  to  be  discussed. 


Figure  113.  a)  Second-generation  AROD  configuration  with  the  NOSC  Hawaii  Lab’s  remote- 
presence  stereo-camera  pair  installed  on  cowling  at  right,  b)  Line  drawing  of  the  portable  AROD 
operator  control  station  (redrawn  from  Martinson,  1988). 
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Ground  Air  TEleRobotic  System  (1985-1987) 

The  GSR,  ATT,  and  AROD  achievements  led  the  Office  of  the  Undersecretary  of  Defense  for 
Tactical  Warfare  Programs/Land  Warfare  (OUSD/TWP/LW)  to  initiate  the  Ground/Air  TEleRobotic 
Systems  (GATERS)  program  in  1985  (Gage,  1995b).  Overseen  by  the  Marine  Corps  Combat 
Development  Command  (MCCDC),  Quantico,  VA,  with  NOSC  serving  as  the  developing  laboratory, 
the  objective  was  to  develop  teleoperated  ground  and  air  systems  to  support  test  and  evaluation  of 
UXS  concepts  by  prospective  military  users.  The  “air”  component  of  GATERS  was  the  Airborne 
Remotely  Operated  Device  (Spain,  1988),  initiated  in  1984  but  eventually  terminated  as  previously 
discussed.  The  “ground”  component  was  a  HMMWV equipped  with  a  0.50-caliber  Weapon  Payload 
and  a  cantilevered  Surveillance  Payload  that  could  be  erected  for  ISR  missions  (Figure  1 14). 


Figure  114.  The  early  Ground/Air  TeleRobotic  System  (GATERS)  prototype  developed  by  Code  531 
in  Hawaii  was  a  remotely  driven  HMMWV  equipped  with  an  elevated  reconnaissance,  surveillance, 
and  target  acquisition  (RSTA)  payload,  with  a  remotely  actuated  0.50-caliber  Browning  M2  machine 
gun  for  self-defense. 

As  explained  by  Hawaii  Lab  researcher  Dr.  Hugh  Spain  (1987),  the  operationally  distinct  ground 
and  air  vehicles  developed  under  the  GATERS  umbrella  shared  several  common  features: 

“Both  are  fiber  optically  tethered.  Both  use  advanced  high-speed  telemetry 
hardware  to  convey  control  and  feedback  signals  back  and  forth  across  the 
fiber  optic  link.  But  perhaps  most  importantly,  both  have  developed  out  of  a 
design  approach  that  emphasizes  the  importance  of  providing  the  human 
operator  with  a  sense  of  telepresence,  an  inside-looking-out  experience  of 
the  remote  system  which  is  intended  to  impart  a  sense  of  being  physically 
present  in  the  vehicle...” 

The  TeleOperated  Vehicle  project  is  further  discussed  in  the  following  section. 
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TeleOperated  Vehicle  (1985-1989) 

The  TeleOperated  Vehicle  (TOV)  concept  emerged  from  the  Ground  Air  TEleRobotic  Systems 
(GATERS)  program  initiated  in  1985.  Built  by  Code  531  at  the  NOSC  Hawaii  Lab  as  previously 
discussed,30  the  baseline  system  consisted  of  a  teleoperated  HMMWV  and  its  associated  operator 
control  station,  connected  by  a  fiber-optic  umbilical  that  provided  secure  high-bandwidth  non-line- 
of-sight  communications  over  distances  up  to  30  kilometers.  Extensive  mobility  testing  of  the  TOV-1 
prototype  was  conducted  in  Hawaii  to  characterize  an  operator’s  ability  to  effectively  negotiate 
challenging  driving  conditions  using  the  remote-presence  stereo-vision  system  (Figure  115). 


Figure  115.  One  of  six  mobility  tests  used  to  establish  a  performance  baseline  for  remote-presence 
teleoperation,  the  Gymkhana  course  featured  1 1  gates,  each  9  feet  wide  (adapted  from  Spain, 
1987).  Gymkhana  is  a  motorsport  in  which  drivers  try  to  achieve  the  fastest  time  possible  over  an 
obstacle  course. 

In  1988,  the  TOV  effort  was  relocated  to  San  Diego  (Figure  1 16a),  where  Code  535  further 
evaluated  and  redesigned  various  components  and  subsystems  in  preparation  for  field  testing  by  the 
USMC  (Figure  1 16b).  The  exploratory  development  and  testing  previously  conducted  by  Code  53 1 
at  the  Hawaii  Lab  was  primarily  focused  on  concept  development,  with  insufficient  time  or  funding 
for  system  hardening.  Under  a  very  tight  program  schedule,  considerable  failure  analysis,  redesign, 
and  fabrication  were  required  in  San  Diego  to  ensure  reliable  performance  during  extended  field 
testing  and  subsequent  operational  demonstrations  at  Camp  Pendleton. 


Figure  116.  a)  Facilities  Manager  Tom  Gaydos,  TOV  team  leader  Robin  Laird,  and  Commander  Bart 
Everett  pull  the  Hawaii-built  TOV-1  prototype  from  its  shipping  crate  upon  arrival  in  San  Diego,  circa 
1988.  b)  Robin  Laird  doubled  as  vehicle  safety  officer  during  preliminary  evaluation  and  hardening  of 
TOV-1  along  Woodward  Road. 


30  See  earlier  Organizational  Structure,  Advanced  Systems  Division  (Code  53)  section. 
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As  with  the  earlier  dune  buggies,  the  TOV operator  was  provided  with  stereo  head-coupled  visual 
displays,  binaural  audio,  and  HMMWV  controls  replicated  in  form,  function,  and  location  to 
minimize  required  operator  training  (Metz,  Everett,  &  Myers,  1992).  Three  operator  control  stations 
and  one  supervisor’s  station  were  housed  in  a  military-standard  Lightweight  Multipurpose  Shelter 
(LMS)  on  the  back  of  another  HMMWV.  Following  extensive  evaluation  at  Camp  Pendleton  (Figure 
116b),  field  trials  of  TOV-1  began  in  May  1988  (Figure  1 17),  to  include  cross-country  transit,  long- 
range  RSTA,  and  remote  firing  of  an  M2  machine  gun.  Meanwhile,  parallel  subsystem  and  vehicle- 
level  testing  to  identify  and  correct  failure  modes  continued  in  San  Diego. 


Figure  117.  TOV-1  undergoes  field  testing  by  Code  535  at  the  Marine  Corps  Tactical  Systems 
Support  Activity  (MCTSSA),  Camp  Pendleton,  CA,  in  1988.  The  TOV  operator  was  provided  with 
stereo  head-coupled  visual  displays,  binaural  audio,  and  driving  controls  isomorphic  to  those  found 
in  a  HMMWV  (Aviles  et  al„  1990). 

Two  head-mounted  microphones  served  as  ears  for  the  robot  in  the  TOV  driver’s  seat  (Figure  117), 
providing  the  operator  with  stereo  hearing  to  heighten  the  remote-presence  effect.  Electric  and 
hydraulic  actuators  for  accelerator,  brakes,  steering,  and  gearshift  were  coupled  via  the  fiber-optic 
tether  to  identical  components  at  the  operator's  station  inside  the  control  van  (Figure  118a).  A  low- 
tension  30-kilometer  cable  payout  system  dispensed  the  control  tether  as  the  vehicle  moved,  avoiding 
the  damage  and  hampered  mobility  that  would  otherwise  arise  from  dragging  the  cable  over  the 
ground  (Aviles  et  al.,  1990).  The  large  joystick  just  right  of  the  steering  wheels  in  Figure  118b 
controlled  the  pan-and-tilt  actuators  for  the  Weapon  and  Surveillance  Modules. 
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Figure  1 18.  a)  Robin  Laird  sits  at  “Operator  Control  Station  1”  inside  the  new  control  van  outfitted  by 
Code  535,  which  was  installed  in  a  HMMWV-mounted  environmental  shelter,  b)  The  shelter  housed 
three  operator  control  stations  (foreground  right  and  background)  and  one  supervisor  station 
(foreground  left). 

The  TOV project  continued  under  the  newly  established  Unmanned  Ground  Vehicle  Joint  Program 
Office  (UGV  JPO)  Ground-Launched  Hellfire  phase  (Metz,  Everett,  &  Myers,  1 992),  with 
Commander  Everett  as  chief  engineer.  TOV- 2,  an  improved  second-generation  version,  was  designed 
and  built  during  this  period  to  support  an  upcoming  milestone  demonstration  in  September  1989.  To 
save  valuable  time,  a  mockup  HMMWV- mounted  shelter  was  constructed  out  of  2x2s  skinned  on 
both  sides  with  plywood  (Figure  116a)  and  covered  in  fiberglass  for  structural  and  watertight 
integrity  (Figure  116b).  This  insulated  air-conditioned  surrogate  shelter  allowed  simultaneous  testing 
and  debugging  at  our  NOSC  Seaside  facilities,  as  well  as  in  the  field  at  Camp  Pendleton. 


Figure  1 19.  a)  A  surrogate  HMMWV  shelter  was  fabricated  by  Commander  Bart  Everett  to  allow 
simultaneous  field  testing  at  our  Northern  Test  Site  as  well  as  at  Camp  Pendleton,  b)  Seaside 
Facilities  manager  Tom  Gaydos  examines  the  recently  fiberglassed  shelter  prior  to  door  installation 
and  painting  (see  also  Figure  120b). 

Three  payload  modules  for  mobility,  surveillance,  and  weapons  allowed  the  TOV  platforms  to  be 
configured  for  a  variety  of  tactical  missions  (Aviles  et  al.,  1990;  Metz,  Everett,  &  Myers,  1992).  The 
Mobility  Module  included  the  necessary  video  cameras  and  actuation  hardware  to  enable  remote 
driving  from  a  standoff  of  several  kilometers.  Two  cameras  on  the  robot  in  the  driver's  seat  (Figure 
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120a)  fed  two  miniature  video  monitors  on  the  operator's  helmet  (Figure  120b),  so  that  the  human 
driver  saw  in  the  van  whatever  the  robot  was  viewing  out  in  the  field  (Everett,  2005).  As  with  the 
A 7T“Dune  Buggy”  project,  the  robot  in  the  remote  HMMWV was  slaved  to  the  operator's  helmet 
back  in  the  control  van  so  as  to  mimic  his  or  her  head  movements  (Martin  &  Hutchinson,  1989). 


Figure  120.  a)  A  Code  531  remote-presence  payload  was  situated  in  the  driver’s  seat  of  TOV-2. 
b)  The  Polhemus  3D  tracking  sensor  on  the  ceiling  of  the  surrogate  shelter  measured  the  angular 
orientation  of  a  3D  inductive  source  attached  to  the  Corporal  Rabidoux’s  helmet.  Captain  George 
Murray  (center)  and  Corporal  Liburd  wait  their  turn. 

The  first-generation  Surveillance  Module,  designed  and  built  by  the  Hawaii  Lab  for  TOV-1,  was  a 
belt-driven  pan-and-tilt  unit  transporting  a  300-pound  high-resolution  reconnaissance,  surveillance, 
target  acquisition  (RSTA)  package  mounted  on  a  cantilevered-lift  mechanism  that  could  raise  it  12 
feet  into  the  air.  The  sensor  suite  consisted  of  a  low-light-level  zoom  camera,  an  AN/TAS-4A  infrared 
imager  (FLIR),  and  an  AN /PA  Q-3  Modular  Universal  Laser  Equipment  (MULE)  designator  (Figure 
121).  The  remote  operator  would  look  for  a  target  with  the  camera  or  the  FLIR,  then  switch  over  to 
the  designator  to  light  it  up  for  a  laser-guided  Hellfire  missile  or  Copperhead  artillery  round. 


Figure  121 .  The  TOV  concept  allowed  RSTA  missions  to  be  carried  out  from  a  vehicle  in  defilade, 
with  a  0.50-caliber  Weapon  Module  for  self-protection  (lower  right).  Validated  targets  were 
illuminated  by  the  MULE  laser  designator  atop  the  Surveillance  Module  (upper  left)  for  a  Hellfire 
missile  or  Copperhead  projectile  strike. 


90 


A  key  disadvantage  of  the  cantilevered  mast  was  the  Surveillance  Module  could  only  operate  in 
the  fully  extended  position,  as  opposed  to  being  raised  just  above  screening  cover  to  minimize 
chances  of  detection.  Figure  122  shows  TOV-2,  a  second-generation  UGV  developed  in  San  Diego 
by  Code  535.  The  main  improvements  in  its  new  Surveillance  Module  were  a  far  more  robust  pan- 
and-tilt  unit,  and  an  electrically  actuated  scissor-lift  mast  that  could  be  raised  to  any  desired  height  up 
to  12  feet  (Spackman,  1989).  The  RSTA-application  payload  was  hardened  for  field  testing,  but 
otherwise  identical  to  that  of  TOV-1,  with  a  MULE  laser  designator  plus  a  FL1R  and  a  conventional 
zoom  camera  for  long-range  surveillance. 


Figure  122.  a)  Code  535’s  second-generation  TO V-2  featured  a  new  Surveillance  Module  (shown 
stowed)  and  Weapon  Module  designed  and  built  by  Manuel  Solorzano.  b)  The  new  Surveillance 
Module  (shown  deployed)  could  be  elevated  to  a  maximum  height  of  12  feet  by  a  scissor-lift 
mechanism  designed  by  Howard  Spackman. 

The  full  range  of  mast  elevation  provided  by  the  new  scissor  lift  facilitated  maintenance  and 
calibration  as  shown  in  Figure  123a,  where  the  MULE  laser  designator  has  been  lowered  to  a  more 
accessible  height  for  boresighting.  This  incremental  process  involved  careful  alignment  of  the  MULE 
designator  with  the  surveillance  camera  to  ensure  reliable  target  illumination  for  Hell/ire  missiles  or 
Copperhead  rounds,  after  which  the  mast  was  raised  for  testing  (Figure  123b).  Part  of  the 
Surveillance  System  redesign  effort  specifically  addressed  acceleration,  moment  of  inertia,  and 
mounting  issues  to  ensure  these  components  did  not  shift  out  of  calibration  during  operation. 


Figure  123.  a)  Robin  Laird  (right)  and  Gunnery  Sargent  Burke  (center)  boresight  the  MULE  laser 
designator  on  the  Surveillance  Module  at  Camp  Pendleton,  CA,  while  Celia  Metz  (lower  left)  confers 
with  Major  McNamee  and  Captain  Murray,  b)  Commander  Everett  supervises  subsequent  MULE 
testing  with  the  mast  fully  extended. 
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The  Weapon  Module  provided  the  UGVs  with  a  remotely  actuated  0.50-caliber  Browning  M2 
machine  gun  for  self-defense  (Figure  124a).  In  addition  to  pan-and-tilt  motion,  electric  actuators 
were  installed  to  charge  the  weapon,  release  the  safety,  and  depress  the  trigger,  all  hardened  second- 
generation  designs  intended  to  eliminate  failure  modes  uncovered  on  TOV-1.  A  fixed-focus  CCD 
camera  (Figure  124b)  was  mounted  just  above  the  gun  barrel  for  safety  purposes.  The  weapon  could 
be  manually  controlled  with  the  joystick  in  response  to  video  from  this  camera,  or  slaved  to  the  more 
sophisticated  electro-optical  sensors  of  the  Surveillance  Module. 


Figure  124.  a)  Commander  Everett  (kneeling)  and  Rick  Smurlo  (foreground)  prepare  to  test-fire  the 
0.50-caliber  machine  gun  of  TOV-1  on  the  firing  line  at  Camp  Pendleton,  circa  1989.  b)  Safety 
Officer  Captain  George  Murray  (seated)  oversees  testing  of  the  new  gun  camera  and  machine-gun 
actuators  prior  to  installation  on  TOV-2. 

In  place  of  the  defensive  machine  gun,  TOV-3  was  to  have  had  a  Rockwell  Hellfire  missile- 
launcher  version  of  the  Weapon  Module.  The  UGV  JPO  decided  to  not  to  fully  outfit  this  vehicle 
with  a  Mobility  Module  or  Surveillance  Module  in  order  to  save  time  and  funding,  so  the  Rockwell 
HMMWV- based  Ground-Launched  Hellfire  (GLH)  system  was  used  instead  (Figure  125).  Another 
causal  factor  was  that  Congressional  language  in  1987  had  restricted  the  use  of  funds  to  acquire  and 
evaluate  new  weapons  mounted  on  robots  (Finkelstein,  2010): 

“A  congressional  staff  member  believed  the  TMAP  platforms  were  too  small 
and  underpowered  to  serve  as  anti-tank  weapons;31  an  urban  legend  spread  that 
Congress  forbid  (sic)  the  development  and  use  of  robots  as  weapons 
platforms.” 

For  this  reason,  the  UGV  JPO  program  manager  decided  to  remove  the  Weapon  Module  from 
TOV-2  just  prior  to  the  end-of-project  demonstrations  in  September  1989. 


31  Two  Teleoperated  Mobile  All-purpose  Platforms  (TMAPs)  prototypes,  built  for  the  US  Army  Missile  Command 
by  Martin-Marietta  and  Grumman  (Weiss  &  Simmons,  1989;  Metz,  Everett,  &  Myers,  1992),  participated  in  the 
Camp  Pendleton  demonstration  in  September  1989.  See  also  Figure  127. 
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Figure  125.  The  Rockwell  HMMWV  with  the  Hellfire  missile  launcher  was  set  up  on  an  adjoining 
hilltop  for  the  final  demonstration  in  September  1989.  Note  Ground-Launched  Hellfire  (GLH)  vehicle 
position  in  Figure  128. 

Celia  Metz  of  Code  535  designed  and  built  the  remote  interface  to  accommodate  the  Ground- 
Launched  Hellfire  system  furnished  by  Rockwell  for  evaluation  and  demonstration.  For  safety 
reasons,  a  dedicated  hard-wire  control  tether  was  run  from  the  control  van  to  the  remote  launch  site 
on  an  adjoining  hilltop  as  shown  in  Figure  126a.  The  underlying  operational  concept  was  that  one 
teleoperated  platform  looked  and  designated  while  the  other  did  the  shooting.  Meanwhile,  all  the 
humans  could  be  up  to  15  kilometers  away,  which  was  important  in  a  number  of  ground- warfare 
scenarios,  particularly  chemical  or  biological. 


Figure  126.  a)  The  Rockwell-furnished  HMMWV  equipped  with  its  Ground-Launched  Hellfire  payload 
was  situated  on  a  nearby  hilltop  at  Camp  Pendleton,  b)  A  Hellfire  missile  comes  off  the  rail  in 
response  to  a  command  from  the  TOV  operator  located  in  the  control  van  several  kilometers  away 
(see  also  map  of  Figure  128). 

Conducted  on  a  live-fire  range  at  Camp  Pendleton,  CA,  the  final  Copperhead  and  Ground- 
Launched  Hellfire  demonstration  was  a  3-day  affair  hosted  by  Lieutenant  Colonel  Bob  Harper  of  the 
UGV  JPO  for  multiple  groups  of  VIP  attendees.  Also  featured  at  this  venue  were  two  prototypes  of 
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the  Teleoperated  Mobile  All-purpose  Platforms  (TMAPs)  built  for  the  U.S.  Army  Missile  Command 
by  Martin-Marietta  and  Grumman  (Weiss  &  Simmons,  1989;  Metz,  Everett,  &  Myers,  1992).  In  one 
of  the  demo  scenarios,  the  Martin-Marietta  TMAP  also  designated  a  target  for  a  Hellfire  missile  strike 
(Figure  127),  which  was  remotely  launched  by  the  TOV operator  in  the  control  van. 


Figure  127.  The  Martin-Marietta  TMAP  (foreground)  participated  in  the  1989  “Tele-Robotic  Vehicle 
Demonstrations”  (see  also  Figure  128),  along  with  the  Grumman  TMAP  (not  shown)  and  the  NOSC 
TOV  (background  left).  The  fiber-optic  cable-recovery  vehicle  developed  by  Code  531  is  at 
background  right. 

To  keep  the  TOV  program  on  schedule,  Code  535  had  routinely  worked  12-  to  16-hour  days,  often 
7  days  a  week,  with  some  taking  just  2  days  off  in  1989  for  Thanksgiving  and  Christmas.  To  counter 
the  stress  and  mental  fatigue,  we  built  a  volleyball  court  south  of  Building  F-36  with  help  from  Tom 
Gaydos,  the  Seaside  facilities  manager  extraordinaire.  The  resulting  exercise  and  camaraderie 
recharged  both  mind  and  body,  the  team  got  its  second  wind,  and  it  was  well  on  its  way  to  becoming 
a  truly  high-performance  organization.  During  the  VIP  demonstrations  that  September  (Figure  128), 
the  TOV  system  achieved  a  perfect  record  of  eight  direct  hits  with  Hellfire  missiles  and  four  direct 
hits  with  laser-guided  Copperhead  projectiles. 
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Figure  128.  Scheme  of  Maneuver  for  the  “Tele-Robotic  Vehicle  Demonstrations”  at  Camp  Pendleton, 
CA,  on  15,  18,  and  20  September  1989.  Note  control  van  adjacent  to  the  observer’s  station  (bottom 
center),  the  Hellfire  launch-vehicle  position  at  lower  right,  and  the  associated  impact  area  at  upper 
left. 

Advanced  Tethered  Vehicle  (1986-1991) 

Designed  and  constructed  by  Code  53  at  the  NOSC  Hawaii  Lab  as  an  improved  follow-on  to  the 
Remote  Unmanned  Work  System  (RUWS),  the  Advanced  Tethered  Vehicle  (ATV)  was  a  purpose-built 
UUV  intended  for  operation  at  depths  as  great  as  20,000  feet  (Murphy,  1991).  Funded  by  the  Naval 
Sea  Systems  Command  (SEA  05R2),  the  ATV  was  considerably  lighter  than  RUWS,  and  could  be 
transported  by  C-130  aircraft  (Lemaire,  1988).  The  system  had  a  forward  speed  of  2  knots,  could 
operate  in  sea  state  3,  and  achieved  a  mean  time  between  critical  failures  of  248  hours  (Morinaga  & 
Hoffman,  1991). 32  As  shown  in  Figure  129,  the  concept  involved  a  tethered  submersible,  various 
surface-handling  components,  two  diesel  generators  for  self-contained  power,  and  an  on-deck  control 
van  (Hoffman,  1991). 


32  The  mean  time  for  shipboard  repairs  was  just  6  hours  (Morinaga  &  Hoffman,  1991). 
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Figure  129.  A  self-contained  system  independent  of  the  host  ship,  the  A  TV  employed  a  vehicle-lift 
line  in  conjunction  with  a  separate  power/communications  tether  (redrawn  from  Morinaga  & 

Hoffman,  1991).  The  lift-line  ram  tensioner  eliminated  snap  loads  during  vehicle  launch  and  recovery 
(Yumori,  1988). 


While  the  RUWS  was  employed  strictly  as  a  testbed,  the  A  TV  was  intended  for  fleet  use  (Lemaire, 
1988).  The  significantly  improved  design,  which  largely  avoided  problems  encountered  on  earlier 
undersea  systems,  incorporated  many  innovative  features  to  improve  reliability  and  simplify  both 
operation  and  maintenance  (Morinaga  &  Hoffman,  1991): 


•  A  pair  of  identical  manipulators 

•  Two  hydraulic  motor/pump  units 

•  Isolated  tool  hydraulics 

•  Redundant  vehicle  electro-optics 

•  Redundant  vehicle-lift  capability 


•  Redundant  cable-traction  drive  motors 

•  Redundant  deck  power  unit  pumps/motors 

•  Redundant  system  power  supply 

•  Separate  lift  line  with  ram  tensioner 


The  A  TV  was  launched  and  recovered  using  a  steel  lift  line  to  avoid  stressing  the  power  and 
communications  tether  (Figure  130),  and  could  thus  be  towed  by  the  host  ship  while  floats  were 
attached  to  the  remote  end  of  the  combination  cable  prior  to  descent.  This  dual-tether  approach  also 
allowed  the  vehicle  to  be  handled  with  power  off,  reduced  stresses  on  the  submerged  tether 
termination,  and  enabled  a  fully  reversible  launch/recovery  procedure  (Hoffman,  1991).  A  lift-line 
ram  tensioner  eliminated  snap  loads  during  vehicle  launch  and  recovery,  and  also  permitted  the 
power  and  communications  tether  to  be  supported  by  the  lift  line  during  deep  operation  to  reduce 
dynamic  loading  (Yumori,  1988). 
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STRENGTH  MEMBER:  Kevlar-49  served  in  4  layers 
with  Mylar  tape  over  each  layer,  1 .050”  O.D. 


NYLON  AXIAL  CORE 


OUTER  JACKET:  Yellow  Hytrel  with  black  stripe. 

1. 309"  0.0. 

BINDER  TAPE:  Double-faced  Neoprene  backed  Nylon 

THREE  POWER  CONDUCTORS:  Hard-drawn  copper 
wires  around  a  nylon  core,  void  filled,  high  density 
polyethylene  insulation,  0  335r  O.D. 


INNER  JACKET:  Natural  TPR.  0  650"  O.D. 

THREE  OPTICAL  CONDUCTORS:  Optical  fibers  are  Sumitomo 
Multimode.  Eight  steel  wires  served  around  each  Tiber,  Jacketed 
with  Hytrel,  0.160"  O.D. 

POLYESTER  CORDAGE  BRAID 

FILLERS:  Interstices  blocked  with  six  nylon  rods  and  DPR 


Figure  130.  This  cross  section  of  the  23,000-foot  ATV  power  and  communications  tether  shows  the 
three  fiber-optic  cables,  three  electrical  conductors  for  high-voltage  power,  and  a  protective  Kevlar 
strength  member  (adapted  from  Morinaga  &  Hoffman,  1991).  See  also  Figure  132. 

The  ATV  electrical  system  (Figure  131)  was  serviced  by  a  pair  of  three-phase  408-volt  diesel 
generators,  each  rated  for  180  kilowatts.  A  30-kilowatt  transformer  stepped  this  incoming  AC  voltage 
down  to  208  and  120  volts  to  power  electronic  equipment,  lighting,  and  air  conditioning  in  the 
control  van.  Similarly,  a  100-kilowatt  transformer  in  the  high-voltage  console  stepped  up  the 
480-volt,  three-phase  input  to  2400  volts  to  minimize  electrical  losses  along  the  23,000-foot 
umbilical  feeding  the  ATV.  On  board  the  submersible,  this  2400-volt  service  was  connected  to  motor 
switching  circuits  controlling  a  pair  of  25-horsepower  electrohydraulic  units,  which  powered  the 
vehicle  thrusters  and  manipulators.  A  5 -kilowatt  step-down  transformer  supplied  power  for  on-board 
electronics,  equipment,  and  lighting  (Hoffman,  1991). 


Figure  131 .  Three-phase  480-volt  AC  power  from  the  diesel  generators  was  stepped  down  to  208 
and  120  volts  for  the  control  van  equipment,  and  stepped  up  to  2400  volts  for  transmission  down  the 
23,000-foot  umbilical  to  the  ATV ,  where  it  was  stepped  down  to  power  the  various  vehicle 
subsystems. 

Instead  of  the  conventional  all-electric  tether  used  on  RUWS ,  the  ATV  umbilical  also  incorporated 
three  fiber-optic  conductors  for  command-and-control  data  plus  video  feedback  (Lemaire,  1988). 
Referring  now  to  the  configuration  depicted  in  Figure  132,  the  telemetry  and  control  interface  inside 
the  control  van  was  connected  to  an  optical  slip  ring  at  the  center  of  the  tether-storage  reel  (see  also 
Figure  129).  Optical  switches  on  the  other  side  of  this  slip  ring  determined  which  of  three  optical 
fibers  in  the  tether  was  used  for  data  transfer  (Hoffman,  1991).  On  board  the  ATV,  each  of  these 
fibers  terminated  in  redundant  electro-optical  circuitry  that  interfaced  with  the  vehicle  electronics. 
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Figure  132.  The  control  van  was  connected  via  an  optical  slip  ring  to  a  three-channel  switch  that 
could  select  any  one  of  three  redundant  optical  fibers  in  the  tether  to  support  the  necessary  full- 
duplex  communication  stream. 


The  two-station  control-van  layout  is  shown  in  Figure  133.  The  primary  vehicle-control  interfaces 
were:  1)  joystick  propulsion  control,  and  2)  input  devices  for  the  dual  force-feedback  manipulator 
controllers.  Sensor  displays  included  a  flat-panel  stereo  TV,  sonar,  and  navigation  (Murphy,  1991). 
The  vehicle-operator  station  on  the  right  side  of  the  control  console  integrated  selectable  real-time 
video  with  graphic  overlays  to  minimize  the  operator’s  required  field  of  view  (Hoffman,  1991).  The 
work-operator  station  was  on  the  left  side  of  the  console,  while  the  tether  operator  sat  at  the  tether- 
control  station  on  the  far  side  of  the  auxiliary  console  (Hoffman,  1991). 
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Figure  133.  The  control  van,  8  feet  wide  by  8  feet  high  by  20  feet  long,  featured  a  centrally  located 
control  console  with  individual  control  stations  for  the  vehicle  operator  and  work  operator  (center),  as 
well  as  a  third  console  for  tether  control  (far  left). 

As  shown  in  Figure  134a,  the  A  TV  frame  was  constructed  from  standard  aluminum  shapes  with 
side  and  front  fairings  to  reduce  drag  and  provide  protection  for  interior  components  (Hoffman, 
1991).  The  work  package  mounted  on  the  aft  end  of  the  vehicle  featured  dual  master-slave 
manipulators,  with  a  stereo-camera  pair  mounted  on  a  pan-and-tilt  unit  to  create  a  3D  video 
representation  of  the  work  space.  Two  additional  television  cameras  were  provided,  one  with  a  zoom 
lens  (Figure  134b)  and  one  that  served  as  a  viewfinder  for  a  35-millimeter  underwater  film  camera 
(Murphy,  1991).  A  fifth  television  camera  and  underwater  sonar  used  during  transit  to  the  work  site 
were  mounted  behind  the  front  fairing  (Hoffman,  1991). 
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Figure  134.  a)  The  underwater  lights  for  the  various  work-package  television  cameras  were  mounted 
on  the  aft  end  of  this  preliminary  version  of  the  neutrally  buoyant  ATV  submersible,  b)  Zoomed-in 
television  image  of  an  object  grasped  by  the  rightmost  master-slave  manipulator. 

The  dual  manipulators  shown  on  the  aft  end  of  the  submersible  in  Figure  135  featured  both 
position  control  and  force  feedback,  with  a  hydraulic  tool  package  that  included  a  spreader,  drill 
motor,  wrench,  and  rotary  saw  mounted  on  a  support  tray  below  the  manipulators.  Syntactic  foam 
modules  were  mounted  beneath  the  top  fairing,  with  hydraulic  manifolds  and  plumbing  in  the  center 
section,  and  titanium  pressure  housings  for  the  electrical  and  optical  components  situated  on  the 
lower  level  (Hoffman,  1991).  The  boom  for  the  power  and  communications  tether  (not  shown)  was 
pivoted  to  enable  horizontal  orientation  while  being  towed  on  the  surface,  and  vertical  once 
submerged. 


Figure  135.  In  this  interim  version  of  the  ATV,  the  horizontal  and  vertical  thrusters  are  shown  on  the 
port-side,  with  the  dual-manipulator  work  package  mounted  on  the  aft  end.  Note  float  on  the  power 
and  communication  tether  (in  water,  lower  left)  and  lifting  cable  at  top  center. 

Deployment  and  recovery  was  accomplished  using  the  small-waterplane-area  twin-hull  (SWATH) 
vessel  SSP  Kaimalino  shown  in  Figure  136  (Hightower  &  Sciple,  1978).  Numerous  test  dives  were 
completed  off  Hawaii  in  1985,  reaching  depths  of  12,000  feet  (Lemaire,  1988).  In  1990,  21  dives 
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totaling  248  hours  of  operation  at  depths  down  to  20,600  feet  were  logged  during  integrated  system 
test  and  evaluation,  with  the  ATV meeting  all  its  reliability  and  effectiveness  objectives  (Hoffman, 
1991).  Extensive  evaluation  of  the  work  package  showed  operators  could  effectively  perform  the 
attachment,  manipulation,  and  tool  operation  tasks  required  for  deep  ocean  work.  Documentation 
validation  and  further  system  evaluation  followed,  conducted  by  the  Undersea  Robotics  Laboratory 
located  Bayside  at  NOSC. 


Figure  136.  Built  in  1973,  the  NOSC  tandem-strut,  small-waterplane-area  twin-hull  (SWATH)  vessel 
SSP  Kaimalino  provided  deployment  and  recovery  services  for  the  ATV  undersea  vehicle  during  test 
and  evaluation  (Lang  &  Slogget,  1985).  The  ATV  was  designed  for  operational  deployment  from  a 
rescue  and  salvage  ship  (ARS). 

In  February  1993,  the  ATV  (Figure  137)  was  transferred  for  fleet  use  to  Submarine  Development 
Group  One  in  San  Diego  (Dziak  et  al.,  1996;  Rona  et  ah,  1997). 33  Here  it  served  as  the  Navy's  west- 
coast  unmanned  deep-water  asset,  complementing  the  Cable-controlled  Underwater  Recovery 
Vehicle  III  (CURV III)  on  the  east  coast.  NOSC  continued  to  provide  support,  engineering  upgrades, 
and  fabrication  of  new  system  components.  In  May  1998,  the  ATV  assisted  the  National  Geographic 
Society  in  locating/photographing  the  WWII  carrier  USS  Yorktown  some  16,650  feet  below  the 
surface  of  the  Pacific  Ocean  (Zumberge,  Sasagana,  &  Spiess,  2006). 


Figure  137.  The  steel  lift  line  is  attached  to  the  lifting  ring  (top  center)  of  this  later  ATV  configuration, 
while  the  power  and  communications  tether  is  shown  at  far  left.  In  addition  to  its  four  video  cameras, 
the  ATV  also  employed  obstacle-avoidance  sonar  and  continuous-transmission  frequency- 
modulated  (CTFM)  sonar  for  target  acquisition. 


33  Now  Submarine  Development  Squadron  5  in  San  Diego,  CA. 
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After  8  years  of  fleet  service,  the  ATV  was  transferred  to  the  Marine  Physical  Laboratory  (MPL)  at 
Scripps  Institute  of  Oceanography,  immediately  adjacent  to  NOSC  on  the  Point  Loma  peninsula  in 
San  Diego.  The  Marine  Physical  Laboratory  and  the  University  of  Hawaii  were  to  operate  and 
maintain  the  UUV  for  scientific  research,  while  the  Navy  retained  ownership.  The  system  remained 
idle  until  2003,  when  the  Office  of  Naval  Research  funded  an  operational  evaluation  down  to  3600 
feet,  supported  by  the  Scripps  research  vessel  RJV  Roger  Revelle,  with  assistance  from  Oceaneering 
personnel  (Zumberge,  Sasagawa,  &  Spiess,  2006).  Following  this  successful  evolution,  however,  the 
ATV  was  demobilized  at  the  Nimitz  Marine  Facility  in  Point  Loma. 

TeleOPerator  Telepresence  System  (1988-1992) 

Never  designed  for  underwater  use,  the  previously  discussed  “Green  Man”  prototype  was  unable 
to  demonstrate  the  master-slave  proficiency  required  by  the  diver-equivalency  concept  of  interest  to 
the  Naval  Sea  Systems  Command  (NAVSEA).  To  address  this  and  other  identified  shortcomings,  the 
second-generation  TeleOperator  Telepresence  System  /  Concept  Verification  Model  (TOPS/CVM) 
program  was  initiated  in  1988.  The  end  goal  was  a  highly  dexterous  telerobotic  master-slave  work 
system  that  could  mimic  the  manipulative  dexterity  of  humans  (Figure  138),  but  without  their 
inherent  operational  limitations  (i.e.,  depth,  endurance,  temperature,  and  exposure).  Overall  systems 
integration  was  performed  by  NOSC  Code  53,  with  key  component  technologies  including  human- 
level  dexterity,  force-reflective  exoskeletal  controllers,  and  stereoscopic  vision. 


Figure  138.  a)  The  TOPS  remote  slave  had  a  single  9-degree-of-freedom  (DOF)  hand  attached  to  a 
7-DOF  arm,  which  was  mounted  on  a  3-DOF  torso,  b)  The  TOPS  operator  used  an  instrumented 
exoskeleton  to  control  the  slave,  based  on  high-resolution  video  feedback  presented  to  the  operator 
on  a  helmet-mounted  stereo  display. 


To  extend  the  work  volume  of  the  manipulator,  a  dexterous  9-DOF  force-reflective  hand  attached 
to  a  7-DOF  force-reflective  arm  was  mounted  on  a  3-DOF  torso.  On  top  of  the  torso  was  a  3-DOF 
head  equipped  with  a  stereoscopic  camera  pair  that  provided  a  70-degree  field  of  view  with  full 
stereo  overlap  (see  Figure  138a).34  The  operator  viewed  the  remote  worksite  in  three  dimensions 


34  The  stereo-vision  system  employed  technologies  developed  by  Wright-Patterson  Air  Force  Base  and  Technology 
Innovation  Group. 
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through  a  high-resolution  (1023-line)  helmet-mounted  display  as  shown  in  Figure  138b.  The 
manipulator,  arm,  torso,  and  head  were  developed  by  Sarcos  Research  Corporation  and  the  Center  for 
Engineering  Design  at  the  University  of  Utah. 

The  high-level  control  and  processing  system  integrated  all  TOPS/CVM  subsystems  under  a 
Center-developed  distributed  heterogeneous  control  paradigm,  which  featured  two  separate  human- 
machine  interfaces.  The  Supervisor  Interface  allowed  technical  personnel  ready  access  to  all  sensing 
and  actuation  subsystems  during  development  and  testing.  The  Operator  Interface  allowed  the  human 
operator  to  easily  control  all  major  system  functions  and  modes  using  voice  commands  with  graphic- 
overlay  feedback.  The  operator  remotely  commanded  the  hand/arm  combinations  through  an 
exoskeletal  controller  (Figure  139),  while  the  torso  and  head  motions  were  automatically  slaved  to 
those  of  the  operator. 


b) 

Figure  139.  a)  This  close-up  of  the  right-hand  exoskeletal  controller  shows  the  degree  of  complexity 
involved  to  achieve  near-human  equivalence  in  remote  manipulation,  b)  The  sophisticated  end 
effector  on  the  slave  could  firmly  grasp  and  operate  conventional  power  tools. 

Modular  Robotic  Architecture  (1 988-1 992) 

The  Modular  Robotic  Architecture  was  developed  as  a  generic  on-board  architecture  offering 
developers  a  standard  set  of  software  and  hardware  tools  that  could  be  used  to  quickly  design 
modular  robotic  prototypes  with  minimum  start-up  overhead  (Smurlo  &  Laird,  1990;  Laird,  Smurlo, 
&  Timmer,  1991).  The  concept  facilitated  customization  of  a  testbed  system  by  providing  sensor, 
actuator,  and  processing  modules  that  could  be  configured  on  demand  as  required  by  the  particular 
needs  of  the  application  being  addressed.  In  a  development  environment,  the  ability  to  later 
incorporate  newer  modules  of  increasing  sophistication  facilitated  evolutionary  growth  potential, 
ensuring  maximum  effective  service  life  before  the  system  became  obsolete. 

An  example  embodiment  under  this  concept,  the  ModBot  (modular  robot)  featured  several 
independent  modules  of  varying  sophistication  interconnected  via  a  generalized  distributed  network. 
The  physical  design  of  the  interlocking-ring  concept  originated  during  earlier  discussions  between 
then  Lieutenant  Commander  Bart  Everett  and  MIT  co-op  student  Anita  Flynn  at  the  Naval  Surface 
Weapons  Center,  White  Oak,  MD,  in  the  summer  of  1986.  The  first  instantiation  at  NOSC  consisted 
of  a  commercial  TRC  LabMate  mobility  base  (Figure  140),  augmented  with  various  sensor,  actuator, 
and  processing  modules  that  enabled  the  robot  to  obtain  and  process  different  information  about  its 
surroundings. 
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Figure  140.  Manufactured  by  Transitions  Research  Corporation,  Bethel,  CN,  the  LabMate  was  used 
as  a  mobility  base  by  a  number  of  research  groups,  and  formed  the  basis  of  the  company’s  later 
HelpMate  robot  (Katevas,  2001).  The  circular  aluminum  base  plate  accepts  the  stacked-ring  ModBot 
payloads  shown  in  Figure  141. 

As  explained  by  project  manager  Robin  Laird  (Babb,  1990): 

“The  idea  of  the  modular  robot  is  similar  to  the  IBM  PC  with  its  expansion  slots; 
adding  a  module  to  the  ModBot  is  like  adding  a  peripheral  card  to  a  PC.  One 
simply  plugs  a  card  into  an  available  slot,  installs  the  supplied  software  drivers, 
and  incorporates  the  new  capabilities  of  the  card  into  the  system.  Adding 
modules  and  capabilities  to  a  ModBot  will  be  just  as  simple,  making  integration 
of  new,  smarter,  better,  faster  modules  very  easy.” 

A  second-generation  adaptation  of  the  irrigation-pipe  housing  employed  on  ROBART II,  the 
stackable  18-inch  rings  for  the  various  modules  were  machined  on  a  lathe  to  interlock  as  shown  in 
Figure  141.  The  24-transducer  Collision- Avoidance  Sonar  Array  was  active  whenever  the  robot  was 
in  motion,  continuously  looking  for  obstacles  within  a  predefined  distance  and  reporting  such  to  the 
High-Level  Processing  Module.  The  Near-Infrared  Proximity  Sensor  Module  provided  a  redundant 
sensing  means  for  obstacle  detection  in  relatively  close  proximity  to  the  robot.  Used  to  complement 
data  obtained  by  the  sonar  array,  this  ring  contained  1 1  Banner  diffuse-mode  optical  proximity 
sensors  facing  the  forward  180  degrees,  each  adjusted  for  a  maximum  detection  range  of 
approximately  3  feet. 
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Figure  141.  a)  Notional  block  diagram  of  the  NOSC  ModBot  concept,  b)  The  interlocking  ring 
modules  employed  on  the  ModBot  were  easily  machined  from  a  length  of  18-inch-diameter  plastic 
water  pipe.  This  configuration  has  a  custom-designed  monocular  pan-and-tilt  unit  built  by  technician 
Steve  Timmer. 


The  High-Level  Processing  Module  (Figure  141a),  which  housed  a  WinSystems  AT286  computer 
mounted  in  a  card  cage  (Figure  142),  used  an  internal  map  representation  and  information  from  other 
modules  to  plan  and  execute  paths  to  desired  waypoints.  This  on-board  module  also  received  external 
commands  from  the  remote  Control  Station  Module,  which  communicated  with  the  ModBot  via  the 
on-board  Communications  Module.  An  RS-232  umbilical  cable  was  used  during  the  early  stages  of 
development,  later  replaced  by  an  OCI  LA  WN  spread-spectrum  RF  link.  Some  exploratory  work  was 
also  performed  using  a  full-duplex,  near-infrared  datalink  made  by  STI.  The  modular  nature  of  the 
robot  and  its  on-board  architecture  allowed  these  Communications  Module  tradeoffs  to  be  quickly 
assessed  without  any  reconfiguration  necessary  for  other  system  modules. 
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Figure  142.  The  High-Level  Processing  Module  of  the  ModBot  housed  one  or  in  some  cases  two 
WinSystems  AT286  computers  mounted  in  a  card  cage  as  shown  at  lower  right  (see  also  Figure 
143).  The  18-inch-ID  ring  sections  in  the  background  are  shown  prior  to  being  machined  to  interlock 
when  stacked  as  shown  in  Figure  141. 

This  flexibility  and  extendibility  of  the  ModBot  architecture  made  it  a  valuable  testbed  for 
expeditious  pursuit  of  new  ideas  and  applications  involving  robot  mobility.  One  of  the  first  was  an 
upgraded  version  of  the  robotic-security  concepts  developed  on  ROBART II.  The  Intrusion  Detection 
Module  was  used  to  detect  human  movement  in  the  vicinity  of  the  robot  and  report  the  perceived 
range  and  bearing  to  a  remotely  located  Control  Station  Module.  This  on-board  subsystem  consisted 
of  ultrasonic,  passive-infrared,  and  microwave  motion  detectors  covering  the  full  360-degree  field  of 
regard.  A  video  motion  detector  in  this  module  also  received  information  from  the  video  sensors  on 
the  Stereoscopic  Pan-and-Tilt  Module  to  determine  if  an  intruder  was  present.  Audio  and  composite- 
video  signals  were  transmitted  back  to  the  operator  via  two  separate  analog  RF  links. 

This  modular  approach  greatly  facilitated  technology  transfer  to  both  Cybermotion  and  ARDEC 
for  the  newly  established  MDARS-Interior  program  (Babb,  1990),  which  employed  the  Cybermotion 
K2A  Navmaster  mobility  base  (Cadwallander  et  al.,  1993).  In  early  1990,  NOSC  engineers  began 
mentoring  their  Army  counterparts  in  all  aspects  of  indoor  robotic  security,  transferring  existing 
6502- based  software  developed  on  ROBART II  for  conversion  to  68HC 11 -based  code  (Everett, 
Everett,  Gilbreath,  &  Tran,  1990).  This  converted  code  was  to  run  on  a  custom  single-board 
computer  built  by  Dr.  Stanley  H.  Smith  of  S.H.  Smith  Associates,  Montville,  NJ,  which  was 
specifically  designed  for  compatibility  with  the  64-pin  Eurocard  connector  used  on  the  K2A 
Navmaster  expansion  bus  (Smith,  1992;  Cadwallander  &  Smith,  1993). 
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Figure  143.  The  ModBot  concept  facilitated  transfer  of  component  technologies  developed  on 
ROBART II  to  ARDEC  engineers  for  use  on  the  MDARS  Interior  program  (Babb,  1990).  Shown  left 
to  right:  Tim  Papayianis  of  ARDEC,  along  with  Steve  Timmer,  Rick  Smurlo,  and  Robin  Laird  of 
NOSC. 

ARDEC  elected  to  adopt  the  modular  payload  design  of  the  ModBot  with  slightly  larger  24-inch 
sections  of  plastic  pipe  (Figure  144b).  This  approach  allowed  each  of  the  payload  modules  to  be 
assigned  to  individual  ARDEC  engineers  for  replication:  1)  video,  2)  acoustic,  3)  microwave, 

4)  passive  infrared,  5)  sonar,  and  6)  security  assessment.  As  a  consequence,  many  of  the  challenges 
associated  with  power  distribution,  on-board  communications,  and  interference  were  inherently 
eliminated.  An  obvious  concern,  however,  was  that  many  of  the  13  localization  strategies  developed 
on  ROBART II  for  use  in  office  environments  (e.g.,  wall  following,  doorway  transit)  would  be 
relatively  ineffective  in  unstructured  warehouse  environments. 


Figure  144.  Shown  here  atop  a  Cybermotion  K-2A  Navmaster  mobility  base,  the  ARDEC  MDARS- 
Interior  payload  was  a  24-inch-diameter  ModBot  configuration  with  six  stacked  modules  for  video, 
acoustic,  microwave,  passive  infrared,  sonar,  and  security  assessment. 
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One  of  the  alternative  localization  technologies  consequently  investigated  by  NOSC  was  a 
guidepath-following  system  used  by  automated  guided  vehicles  (AGVs)  in  similar  industrial 
scenarios.  Rather  than  rely  entirely  upon  guidepath  following,  the  envisioned  MDARS  approach  was 
to  use  a  hybrid  navigation  scheme  featuring  sections  of  guidepath  “freeways”  installed  only  in  those 
areas  deemed  difficult  for  off-path  navigation  (Holland,  Everett,  &  Gilbreath,  1990).  The  initial 
prototype  tested  for  MDARS  was  inspired  by  the  use  of  a  Litton  guidepath  sensor  employed  on  a  Bell 
and  Howell  mail-delivery  cart  (Figure  145a),  which  had  been  operational  for  years  in  Building  A33 
Topside  at  NOSC  (Everett,  1995a). 


Figure  145.  a)  Shown  here  on  a  mail-delivery  cart  in  Building  A33,  the  Litton  guidepath  sensor 
followed  a  transparent  chemical  stripe  that  visibly  glowed  due  to  secondary  emissions  when 
irradiated  by  ultraviolet  light  (Everett,  1995a).  b)  An  alternative  NOSC  design  employed  an  array  of 
four  Banner  retroreflective  sensors. 

A  less  expensive  alternative  to  the  Litton  stripe-follower  was  investigated  for  this  purpose,  in 
which  a  retroreflective  tape  was  tracked  by  an  array  of  four  Banner  near-infrared  sensors  (Figure 
145b).  This  approach  was  rejected  due  to  anticipated  guidepath  degradation  in  industrial  settings.  The 
Litton  guidepath  was  far  superior  in  this  regard  to  the  retroreflective  tape,  in  that  it  was  nearly 
invisible  to  the  eye  (except  under  ultraviolet  light)  and  held  up  well  over  time.  Accordingly,  the 
Reconfigurable  Video  Line  Digitizer  software  developed  for  ROB  ART  II  was  modified  to  digitize  the 
output  of  a  CCD  camera  viewing  the  illuminated  chemical  guidepath  on  the  floor  surface 
immediately  ahead  of  the  ModBot  (Figure  146). 
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Figure  146.  Originally  developed  for  video  motion  detection  as  a  security  sensor,  this  block  diagram 
shows  the  Reconfigurable  Video  Line  Digitizer  used  to  digitize  video  lines  60,  120,  and  180  for 
tracking  secondary  emissions  from  the  UV-stimulated  chemical  guidepath.  Barcoded  position 
markers  adjacent  to  the  stripe  could  also  be  read. 

For  testing  purposes,  masking  tape  performed  admirably  as  a  temporary  guidepath  when  irradiated 
by  an  ultraviolet  source.  Developed  by  Theresa  Tran,  the  new  software  determined  the  real-time 
lateral  offset  and  orientation  of  the  guidepath  from  three  digitized  video  lines  as  shown  in  Figure  147. 
The  white  rectangle  in  the  image  outlines  the  camera  field  of  view,  while  the  three  horizontal  lines  of 
the  skewed  figure-8  pattern  reflect  guidepath  detection  by  the  three  video  scan  lines.  The  inherent 
modularity  of  the  ModBot  architecture  greatly  facilitated  this  component  test  and  evaluation  in 
parallel  with  ongoing  MDARS  development,  without  tying  up  any  MD^R^-specific  resources.  The 
guidepath  “freeway”  concept,  however,  was  found  to  be  unnecessary  in  later  warehouse  installations 
of  MDARS-Interior. 


Figure  147.  Screenshot  showing  horizontal  plot  of  digitized  video  lines  60,  120,  and  180  from  the 
Reconfigurable  Video  Line  Digitizer  developed  under  the  ROBART II  Independent  Exploratory 
Development  effort.  The  robot’s  lateral  offset  and  orientation  with  respect  to  the  guidepath  stripe  as 
calculated  by  the  software  appear  at  lower  left. 

Another  example  of  the  ModBot  facilitating  rapid  response  to  customer  needs  involved  hazardous- 
waste  inspection  for  the  U.S.  Department  of  Energy  (DoE)  at  the  Savanah  River  Site  in  Aiken,  SC. 
The  Center  was  asked  to  evaluate  potential  navigational  referencing  sensors  for  this  new-start  robotic 
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effort  to  automatically  inspect  stored  55-gallon  drums  of  nuclear  waste.  The  drums  were  typically 
packed  four  to  a  standard  shipping  pallet,  which  in  turn  was  stacked  three  high  and  arranged  in  rows. 
Potential  mobility  platforms  under  consideration  were  the  Cybermotion  K3A  Navmaster  and  the  TRC 
LabMate.  This  application  was  further  complicated  because  the  access  aisles  among  the  rows  of 
stacked  drums  were  very  narrow,  leaving  very  little  room  for  navigational  errors. 

One  of  the  tasks  addressed  by  NOSC  was  detecting  the  center  of  each  drum  so  the  robot  would  be 
appropriately  positioned  to  capture  a  high-resolution  position-stamped  video  image,  which  was  later 
post-processed  to  identify  any  barrel  deterioration  or  leakage.  The  ModBot  was  quickly  configured 
with  the  new  optical  proximity-sensor  ring  shown  in  Figure  148a  to  complement  its  sonar-array  ring. 
This  approach  allowed  simultaneous  testing  of  both  sensor  options  on  a  common  drum  target,  which 
made  cross  correlation  of  the  results  far  more  relevant  and  timely.  Cybermotion,  meanwhile,  was 
brought  under  contract  by  DoE  to  build  a  prototype  visual-inspection  payload  that  could  image  all 
three  drums  in  the  stack  (Figure  148b). 


Figure  148.  a)  Shown  here  with  electrical  engineer  Theresa  Tran,  the  ModBot  was  quickly 
reconfigured  with  a  variety  of  near-infrared  proximity  sensors  for  individual  evaluation  in  a  drum- 
detection  role,  b)  The  vision-based  drum-inspection  payload  atop  this  K3A  Navmaster  base  was 
developed  for  DoE  by  Cybermotion,  Inc. 

Another  configuration  of  the  ModBot  served  as  a  ready  testbed  for  the  Sensor-Motor 
Transformation  research  project,  which  applied  biological  neural  modeling  to  autonomous  robotic 
control  (Figure  149a).  As  all  processing  and  software  development  took  place  on  the  ModBot  itself, 
an  additional  High-Level  Processing  Module  was  added  to  house  an  80486- based  computer  hosting 
two  Intel  i860  co-processors  and  a  120-megabyte  hard  drive.  Sensor  input  consisted  of  data  from  the 
video  camera  (Figure  149b)  and  wheel  encoders,  all  other  ModBot  sensor  modules  having  been 
removed  or  disconnected.  This  effort  is  discussed  in  further  detail  in  the  Project  Summaries  section 
of  this  volume  entitled  Sensor-Motor  Transformation. 
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Figure  149.  a)  The  ModBot  configured  with  a  Sonar,  Proximity,  and  two  High-Level  Processing 
Modules  to  support  the  Robotic  Sensor-Motor  Transformation  payload,  circa  1990.  b)  Close  up  of 
the  video  camera  and  laser  pointer  on  a  video  pan-and-tilt  unit  built  by  technician  Steve  Timmer. 

The  stacked-ring  concept  employed  on  the  ModBot,  while  well  suited  for  research,  development, 
and  evaluation,  was  clearly  not  practical  for  production  systems,  as  there  are  far  better  ways  to 
optimize  size  and  weight.  In  retrospect,  however,  there  were  a  number  of  benefits  realized,  many  of 
which  were  not  always  fully  appreciated  at  the  time,  that  significantly  influenced  the  path  forward. 
The  most  obvious  of  these  was  the  ability  to  rapidly  respond  to  an  emergent  customer  or  sponsor 
need,  as  illustrated  by  the  DoE  nuclear-inspection  query.  Equally  apparent  was  the  facilitation  of 
technology  transfer,  as  reflected  in  the  transition  of  robotic-security  hardware  and  software  from 
ROB  ART  II  to  the  U.S.  Army  via  the  ModBot. 


Figure  150.  a)  The  pepper-ball  gunpod  payload  was  attached  to  a  standard  mounting  plate  atop  the 
MDARS-Exterior  security  robot,  which  is  further  discussed  in  a  later  section,  b)  The  vertical-takeoff 
launch-and-recover  payload  quickly  attached  to  the  same  standard  mount.  See  also  Figure  151. 
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The  long-term  impact  was  subtle  but  far  more  significant.  As  the  term  ModBot  implies,  the 
underlying  concept  further  encouraged  a  modular  approach  that  influenced  everything  the  Advanced 
Technology  Development  Branch  undertook.  The  numerous  payloads  developed  for  MDARS- 
Exterior  shared  a  standardized  mounting  scheme,  as  evidenced  by  the  various  gun  pods  (Figure  150a) 
and  UAV  launch/recover  payloads  (Figure  150b)  on  top,  and  the  marsupial  UGV  carrier  (Figure 
151a)  and  UA  V refueling  payload  on  the  back  (Figure  151b).  This  modular  mindset  was  key  to  the 
subsequent  development  of  the  Multi-robot  Operator  Control  Unit  (MOCU)  command-and-control 
and  Autonomous  Capabilities  Suite  (ACS)  on-board  architectures,  both  of  which  facilitated  our  later 
Interoperability  Profile  support  for  the  RS  JPO. 


Figure  151 .  a)  The  tracked  Man-Portable  Robotic  System  robot  is  shown  descending  from  the 
marsupial  carrier  on  the  back  of  the  MDARS-Exterior  (BAA-version)  robot,  b)  The  Autonomous  UAV 
Mission  System  (AUMS)  refueling  payload  attaches  to  the  same  standardized  mounts  on  the  top  and 
rear  of  the  MDARS  platform. 

CHRONOLOGICAL  PROJECT  SUMMARIES  (1990-1999) 

In  1992,  the  Naval  Ocean  Systems  Center  (NOSC)  was  renamed  the  Naval  Command  Control  and 
Ocean  Surveillance  Center  (NCCOSC),  which  later  became  the  Space  and  Naval  Warfare  Systems 
Center  San  Diego  (SSC  San  Diego)  in  1997.  In  1993,  the  Advanced  Technology  Development 
Branch  (Code  535)  was  reorganized  as  the  Adaptive  Systems  Branch  (Code  531),  with  the 
organizational  code  later  changed  from  Code  531  to  Code  D371  in  1996  (Appendix  E). 

Surrogate  Teleoperated  Vehicle  (1990-1992) 

Concerned  by  a  proliferation  of  apparently  uncoordinated  DoD  development  efforts,  Congress 
mandated  as  part  of  the  FY  1990  Defense  Appropriations  Bill  that  all  UGV  projects  be  consolidated 
under  the  policy  and  program  direction  of  the  Office  of  the  Secretary  of  Defense  (Metz,  Everett,  & 
Myers,  1992).  To  facilitate  this  goal,  the  Unmanned  Ground  Vehicle  Joint  Program  Office  (UGV 
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JPO)35  had  been  stood  up  in  Huntsville,  AL,  in  1988  as  the  central  focal  point  for  the  development 
and  fielding  of  U.S.  Army  and  Marine  Corps  UGV  systems  (Anderson,  2014).  A  key  element  of 
UGV  JPO  strategy  involved  near-term  fielding  of  multiple  testbed  vehicles  that  could  allow  users  to 
generate  and  refine  operational  concepts. 

To  help  achieve  this  goal,  the  TOV portion  of  GATERS  transitioned  to  the  Surrogate  Teleoperated 
Vehicle  (STV)  program  in  1990.  Overseen  by  the  UGV  JPO,  the  STV  (Figure  152)  was  developed 
under  contract  to  the  Naval  Command  Control  and  Ocean  Surveillance  Center  (NCCOSC,  formerly 
NOSC)  by  Robotic  Systems  Technology  (RST),  Westminster,  MD.36  Fourteen  delivered  vehicles 
enabled  large  numbers  of  military  personnel  to  gain  valuable  hands-on  UGV  experience  that  could 
positively  influence  subsequent  acquisition  strategies  (Metz,  Everett,  &  Myers,  1992). 37  The  STV  was 
designed  small  enough  to  be  helicopter-  and  i/MMJFF-transportable,  yet  large  enough  to 
accommodate  a  human  driver,  with  sufficient  speed  to  keep  up  with  a  tactical  convoy  (35  mph). 


Figure  152.  Conceptual  line  drawing  of  the  Surrogate  Teleoperated  Vehicle  designed  and  built  by 
RST  under  contract  to  the  NCCOSC,  San  Diego,  CA,  and  overseen  by  Tom  Bamburg  (adapted  from 
Myers,  1991). 

The  system  architecture  included  four  principal  subsystems:  1)  the  remote  platform,  2)  the 
Mobility/RSTA  module,  3)  the  communications  system,  and  4)  the  operator  control  unit  (Myers, 
1991).  Based  upon  a  commercial  Polaris  Industries  Big  Boss  six- wheel-drive  all-terrain  vehicle 
measuring  1 17.5  inches  long  and  50.5  inches  wide  ,  the  remote  platform  could  traverse  standing 
water  up  to  2  feet  deep  (Myers,  1992).  A  three-cylinder,  25-horsepower  diesel  engine  built  by  Fuji 
Heavy  Industries  powered  the  Ackerman-steered  vehicle  at  speeds  up  to  35  mph  (Gage,  1995b). 
Coupled  to  the  gearbox  input  shaft  via  an  electric  clutch  (Figure  153),  an  auxiliary  3-horsepower 
electric  golf-cart  motor  enabled  extremely  quiet  movement  during  surveillance  operations  at  reduced 
speed  (4  mph). 


35  Later  redesignated  as  the  Robotic  Systems  Joint  Program  Office  (RS  JPO). 

36  Now  General  Dynamics  Robotic  Systems  (GDRS). 

37  Additional  vehicles  with  a  different  paint  scheme  were  delivered  to  the  French  (Del  Giorno,  2015). 
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Front  Drive  Spindle 


Figure  153.  Normally  powered  by  a  25-horsepower  diesel  engine,  the  STV  could  switch  to 
3-horsepower  electric  drive  for  covert  operation  at  reduced  speeds  up  to  4  mph  (adapted  from 
Myers,  1991). 

In  similar  fashion  to  its  TOV  predecessor,  the  mobility/RSTA  module  consisted  of  a  number  of 
reconnaissance,  surveillance,  and  target-acquisition  sensors  mounted  on  a  pan-and-tilt  mechanism 
situated  atop  an  extendable  scissor-lift  mast.  In  its  stowed  configuration,  the  mast  was  only  24  inches 
high  (Figure  154a),  but  could  raise  the  sensor  pod  to  a  full  height  of  15  feet  above  ground  level 
(Figure  154b).  Adjustable  pneumatic  springs  in  the  rear  of  the  vehicle  provided  for  stiffening  the 
suspension  when  the  surveillance  mast  was  elevated,  thus  reducing  sway  and  jitter  during  RSTA 
operations  (Metz,  Everett,  &  Myers,  1992). 


Figure  154.  a)  The  STV  turns  west  off  Woodward  onto  Robart  Road  during  onsite  testing  in  San 
Diego,  CA.  b)  Shown  here  with  its  RSTA  Module  partially  elevated,  the  STV  was  controlled  by  a 
motorcycle-type  steering  device  (inset  lower  right)  on  the  operator  control  unit  (image  courtesy 
General  Dynamics  Robotic  Systems). 
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Referring  now  to  Figure  155,  the  mobility/RSTA  sensors  included: 

•  A  stereo  pair  of  460-line  daytime  driving  cameras 

•  An  image-intensified  camera  pair  for  nighttime  driving 

•  A  day  targeting  camera  equipped  with  a  14-to-l  zoom  lens 

•  An  image-intensified  night  targeting  camera  with  a  10-to-l  zoom  lens 

•  An  IRIS-T  FLIR  (forward-looking  infrared  camera) 

•  Either  an  LTM-86  laser  ranger/designator  or  an  ESL-100  eye-safe  laser  ranger 


Figure  155.  a)  A  pair  of  olive  drab  STVs  prior  to  delivery  to  the  French,  b)  The  zoom  color  and 
image-intensified  cameras  share  an  enclosure  on  top  left  of  the  tilt  axis  assembly,  with  an  ESL-100 
laser  rangefinder  and  acoustical  array  at  top  right.  The  color  and  image-intensified  stereo  pairs  are 
mounted  on  the  bottom  (Del  Giorno,  201 5). 38 

The  operator  control  unit  employed  a  motorcycle-type  steering  device  for  speed  and  directional 
control,  with  a  two-degree-of- freedom  joystick  for  camera  pan  and  tilt.  The  STV  communications 
system  allowed  the  vehicle  to  be  controlled  from  the  man-portable  operator  control  unit  using  either  a 
deployed  fiber-optic  tether  or  a  back-up  RF  link  (RST,  1993).  The  10-kilometer  inside-wound  fiber¬ 
optic  spool  was  packaged  in  a  3. 5 -cubic-foot  cargo  box  behind  the  engine  compartment,  with  a 
hinged  lid  for  easy  access  (Myers,  1992).  A  low-tension  payout  scheme  fed  the  2. 5 -millimeter  cable 
out  the  back  as  the  vehicle  moved  forward. 

The  RF  back-up  communications  system  consisted  of  the  following  components  (RST,  1993): 

•  A  9600-baud  full-duplex  (dual-frequency)  Repco  SLQ-96  Radio  Modem  for  command  and 
status  data 

•  A  Repco  Utility  Data  System  (UDS)  FM  transmitter  for  audio  to  the  vehicle 

•  A  Dell- Star  Technologies  900-Series  video  transmitter  for  video  and  audio  from  the 
vehicle  to  the  operator  control  unit 

The  maximum  effective  operating  range  on  level-terrain  in  RF  mode  was  approximately 
2  kilometers. 


38  Normally  mounted  beneath  the  tilt  axis  next  to  the  stereo  pairs,  the  IRIS-T  FLIR  was  not  delivered  to  the  French 
(Del  Giorno,  2015). 
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In  February  and  March  1992,  the  first  STV  produced  was  fielded  with  a  group  of  soldiers  and 
Marines  in  a  Concept  of  Employment  Exercise  (COEE)  at  Fort  Hunter-Liggett,  CA  (Figure  156). 
While  the  unmanned  vehicle  demonstrated  it  could  maneuver  well  in  heavily  wooded  areas  and  on 
muddy  slopes,  it  could  not  traverse  deep  ditches,  and  the  RSTA  package  used  the  full  payload 
capacity  of  the  platform,  which  limited  future  flexibility.  On  the  positive  side,  it  was  confirmed  that 
“remote  presence”  visual  displays  such  as  stereo-vision  and  pitch-and-roll  icons  facilitated  driving  at 
higher  speeds  and  on  steeper  side  slopes  by  providing  an  enhanced  sense  of  spatial  and  geographic 
awareness  (Metz,  1992). 


Figure  156.  The  remotely  operated  STV  fords  a  shallow  stream  at  Fort  Hunter-Liggett,  CA,  during 
the  Concept  of  Employment  Exercise  (COEE)  in  1992. 

Mobile  Detection  Assessment  Response  System-interior  990-2003) 

DoD  awareness  generated  by  the  1989  USMC  GATERS demonstration,  as  well  as  continuing 
publicity  garnered  by  ROBART II,  resulted  in  the  U.S.  Army  asking  NOSC  to  support  the  Mobile 
Detection  Assessment  Response  System  (MDARS)  robotic-security  program  of  record.  The  ROBART 
series  of  laboratory  surrogates  (Figure  157)  directly  addressed  the  desired  MDARS  functionality 
reflected  in  the  program  name  as  follows  (Davis,  2011): 

•  Detection:  ROBART  I  could  detect  a  potential  intruder  (Everett,  1982). 

•  Assessment:  ROBART II  could  both  detect  and  assess,  thereby  reducing  the  number  of 
nuisance  alarms  (Smurlo  &  Everett,  1992). 

•  Response:  ROBART  III  (see  next  section)  demonstrated  the  feasibility  of  automated  response, 
using  a  surrogate  Gatling-type  nonlethal  weapon  (Everett  &  Gage,  1996). 
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Figure  157.  The  ROBART series  of  laboratory  surrogates  developed  the  component  technologies 
required  for  the  Mobile  Detection  Assessment  Response  System  (MDARS)  project,  the  start  of 
continuous  funding  from  the  U.S.  Army  program  office  for  some  15  physical-security  and  force- 
protection  projects  through  2014. 39 

Funded  by  the  Physical  Security  Equipment  Management  Office  (PSEMO),  Fort  Belvoir,  VA,40 
the  MDARS-Interior  program  goal  was  to  provide  multiple  robotic  platforms  performing  random 
patrols  within  DoD  warehouses  to  detect  intruders  and  monitor  inventory  status  (Pransky,  1997). 

Two  commercially  available  robotic  platforms  were  purchased  and  evaluated  for  this  application: 

1)  the  K2A  Navmaster  mobility  base  developed  by  Cybermotion,  Inc.  of  Roanoke,  VA,  for  material¬ 
handling  applications  (Figure  158a);  and  2)  the  Sentry  robot  developed  by  Denning  Mobile  Robots  of 
Woburn,  MA,  for  indoor  security  applications  (Figure  158b). 


39  The  Army  Product  Manager,  Force  Protection  Systems  (PdM-FPS)  at  Fort  Belvoir  (formerly  PM-PSE,  formerly 
PSEMO)  has  funded  SSC  Pacific  for  over  25  years  on  a  variety  of  physical-security/force-protection  applications. 

40  PSEMO  was  headed  at  the  time  by  Lieutenant  Colonel  Larry  Petcu,  with  Jerry  Edwards  serving  as  lead  project 
officer  for  MDARS. 
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Figure  158.  a)  The  K2A  Navmaster  robot  was  used  as  a  mobility  platform  by  a  number  of  robotic 
researchers.  The  plywood  turret  extension  (upper  right)  was  built  by  Commander  Everett  to 
temporarily  house  potential  payload  components  for  evaluation,  b)  The  4-foot-tall  Denning  Sentry 
robot  is  shown  here  inside  its  recharging  booth. 

While  the  Denning  Sentry  was  equipped  with  many  of  the  same  motion-detection  sensors  as 
ROB  ART  II  (Figure  159),  no  fusion  of  their  output  states  was  performed  to  increase  the  probability  of 
detection  and  reduce  nuisance  alarms.  The  company’s  developmental  focus  had  been  indoor 
navigation  using  a  vision-based  homing  scheme  that  tracked  near-infrared  LED  beacons  hidden  in 
the  walls  of  an  office  environment  (Kadonoff,  1994),  which  would  have  been  problematic  in 
warehouse  scenarios.  The  Cybermotion  Navmaster  had  no  security  sensors  (the  company  later  added 
a  video  camera),  but  its  navigation  scheme  was  far  more  robust  and  required  no  substantial 
modifications  to  the  operating  environment,  so  it  consequently  became  the  mobility  base  of  choice 
(Babb,  1990). 


Figure  159.  Shown  here  next  to  ROBART II  in  the  museum  area  of  Building  624  Seaside,  the 
Denning  Sentry  employed  microwave  and  passive-infrared  motion  sensors  similar  to  those  used  on 
ROBART  II.  Denning  began  development  of  the  Sentry  in  August  1983  (Everett,  1998). 
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NOSC  provided  MDARS  with  second-generation  collision-avoidance,  path-planning,  and  security- 
assessment  payloads  adapted  from  systems  previously  developed  on  ROBART II.  To  expedite 
progress  and  reduce  risk,  two  different  prototypes  were  pursued,  one  in  Building  F-36  Seaside  at 
NOSC  in  San  Diego  as  shown  in  Figure  160a.  The  other  version  (Figure  160b),  employing  various 
component  technologies  furnished  by  NOSC,  was  assembled  by  Army  engineers  at  Building  30  of 
the  Armament  Research  Development  and  Engineering  Center  (ARDEC),  Picatinny  Arsenal,  NJ 
(Cadwallander,  1993).  Both  these  prototypes  employed  the  Cybermotion  K2A  Navmaster  material¬ 
handling  mobility  base  as  shown  below. 


Figure  160.  a)  The  NOSC  MDARS-Interior  prototype  featured  video,  microwave,  passive-infrared, 
and  sonar  motion  sensors,  circa  1992.  b)  The  NOSC  collision-avoidance  sonar  upgrade  is  mounted 
below  the  turret  of  the  ARDEC  prototype,  with  ARDEC’s  stacked-ring  security-assessment  payload 
above. 

To  facilitate  technology  transfer  from  NOSC,  the  physical  configuration  of  the  ARDEC  security- 
assessment  payload  was  based  on  interlocking  plastic-ring  sections  per  the  previously  discussed 
ModBot  concept  (see  again  Figure  160b).  Note  the  three  microphones  of  the  passive  acoustic- 
detection  array,  originally  developed  on  ROBART II,  mounted  on  the  video  pan-and-tilt  unit  at  the 
top  of  the  robot.  NOSC  also  provided  duplicates  of  ROBART  II’ s  camera  pan-and-tilt  software,  video 
motion-detection  system,  and  the  collision-avoidance  proximity  and  sonar  arrays  (see  block  diagram 
of  Figure  161).  ARDEC’s  tasking  involved  converting  the  6502  assembly-language  software  to 
68HC1 1  code  and  repackaging  the  hardware  (Bradley  et  al.,  1993). 
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Figure  161.  Block  diagram  of  the  MDARS-Interior  functionality  integrated  into  the  K2A  Navmaster 
architecture  via  Cybermotion’s  existing  Supervisory  Link  and  Control  Link  buses. 

To  help  introduce  the  MDARS-Interior  program  to  prospective  users,  a  fiberglass  mockup  of  the 
NOSC  prototype  vehicle  was  constructed  for  PSEMO  by  Commander  Everett  (Figure  162).  All 
external  accessories  (decals,  latches,  sonar  transducers,  the  docking  port,  and  the  beacon-sensor 
assembly)  were  purchased  from  Cybermotion.  In  anticipation  of  rough  handling  during  transport, 
tandem  wheel  assemblies  were  employed  to  minimize  structural  problems,  but  otherwise  the  mockup 
was  a  high-fidelity  replica  of  the  real  thing.  (Cybermotion  later  incorporated  tandem-wheel  drive  on 
their  K3A  platform  upgrade,  as  shown  in  Figure  175.) 


Figure  162.  Outfitted  with  a  surplus  prototype  of  the  security-assessment  payload,  this  lightweight 
fiberglass  mockup  of  the  Cybermotion  K2A  Navmaster  vehicle  was  constructed  on  weekends  by 
Commander  Everett  in  his  garage  workshop  to  provide  PSEMO  with  a  static  display  for  use  at 
conferences  and  trade  shows. 


119 


In  support  of  the  MDARS  inventory-assessment  mission,  a  radio-frequency  identification  (RFID) 
payload  consisting  of  the  Amtech  All 200  Reader  and  AR2200  RF  Module  was  tested  by  ARDEC  on 
a  second  K2A  Navmaster  robot  (Eng,  Lu,  &  Bradley,  1992).  The  initial  Technical  Feasibility  Test 
(TFT-I)  of  the  MDARS-Interior  system,  conducted  in  Building  30  at  Picatinny  Arsenal,  primarily 
focused  on  mobility  and  security  assessment,  and  did  not  address  RFID  product  assessment.  The 
ROBART II  video-line  motion  detector  proved  to  be  the  most  reliable  security  sensor  tested  during 
this  evaluation.  An  alternate  product-inventory  payload  was  evaluated  by  NOSC  in  San  Diego,  using 
Savi  RFID  tags  (Figure  163)  and  interrogators  (Figure  164a)  interfaced  to  the  NOSC-developed 
Product  Assessment  Database  (Smurlo  et  al.,  1995). 


a) 


IWD 


Figure  163.  a).  The  Savi  TyTag  (left)  and  its  SealTag  derivative  (right),  which  was  essentially  a 
TyTag  modified  to  accept  and  report  external  binary  inputs  in  addition  to  its  tag  ID.  b)  A  number  of 
Savi  TyTags  were  attached  to  empty  cardboard  boxes  distributed  throughout  a  warehouse 
environment  for  RFID  evaluation. 


In  March  1992,  NOSC  entered  into  a  cooperative  research  and  development  agreement  (CRADA) 
with  Cybermotion  to  add  a  security  payload  to  the  company’s  K2A  Navmaster  robot.  The  CRADA 
objective  was  to  transfer  ROBART II’s  intelligent  security-detection-and-assessment  software  to 
Cybermotion  so  the  company  could  reproduce  it  in  a  more  cost-effective  configuration  that  could 
become  a  commercial  product  (Babb,  1990).  Rather  than  employ  a  multi-sensor  staring  array  as  used 
by  the  government  to  demonstrate  MDARS  feasibility,  the  very  elegant  Cybermotion  solution  rotated 
a  pair  of  microwave  and  passive-infrared  sensors  to  achieve  the  required  360-degree  coverage  with 
significantly  reduced  size  and  cost  (Figure  164). 
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Figure  164.  a)  The  initial  Savi  RFID  tag  reader  (silver  dome)  is  atop  the  turret  behind  the  camera  pan 
and  tilt  unit,  underneath  the  Cybermotion  Security  Patrol  Instrumentation  (SPI)  module,  b)  In  1994,  a 
second  MDARS  prototype  was  installed  in  the  NOSC  excess-property  warehouse  at  Camp  Elliott, 
after  which  the  two  robots  operated  24/7. 

In  addition  to  providing  component  technologies  to  enhance  the  NavMaster,  NOSC  was  tasked  by 
PSEMO  with  developing  a  command-and-control  architecture  for  coordinated  oversight  of  multiple 
robots.  Prior  to  this  point  in  UGV  history,  for  every  mobile  robot  there  was  some  dedicated  remote 
computer  serving  as  its  one-on-one  operator  interface.  The  MDARS  concept  envisioned  up  to  32 
security  robots  at  any  given  installation,  however,  and  that  many  stove-piped  control  stations  at  the 
site’s  guard  post  was  clearly  impractical.  A  high-level  conceptual  design  for  a  modular  robotic 
architecture  that  could  handle  this  workload  was  subsequently  conceived,  possibly  the  first  ever 
instantiation  of  multiple-robot  control  (Everett,  Gilbreath,  &  Laird,  1992). 

Implemented  by  Gary  Gilbreath  and  Robin  Laird,  the  command-and-control  solution  provided 
supervisory  oversight  of  intelligent  semi-autonomous  security  robots  geographically  distributed 
across  the  area  of  coverage.  The  origins  of  this  approach  trace  back  to  then  Lieutenant  Commander 
Everett’s  ROB  ART  I  thesis  at  the  Naval  Postgraduate  School  (Everett,  1982): 

“A  central  monitoring  station  could  be  alerted  via  a  digitally  coded  radio 
transmission  that  an  intruder  had  been  detected,  subsequently  setting  off  the 
building  security  alarm,  and  possibly  notifying  police.  This  monitoring 
station  could  perhaps  keep  track  of  several  robots  patrolling  on  different 
floors,  or  in  different  areas  of  a  large  industrial  plant.  Each  robot  could 
periodically  transmit  its  location  and  status,  thus  providing  a  means  for  the 
central  station  to  be  alerted  should  a  robot  become  disabled.  Human  guards 
or  other  robots  could  then  be  dispatched  to  the  scene  to  evaluate  the 
situation.” 

Specifically  designed  to  meet  this  objective  (Everett,  Gilbreath,  Laird,  &  Heath-Pastore,  1993; 
Everett,  Gilbreath,  and  Heath,  1993),  the  Multiple  Robot  Host  Architecture  (MRHA)  was  a  distributed 
processing  system  intended  to  run  automatically  with  minimal  human  supervision  (Figure  165). 

Guard  intervention  was  required  only  when  a  patrolling  robot  encountered  an  unusual  condition  such 
as  an  environmental  hazard  or  a  security  breach.  Such  “exceptional  events”  were  prioritized  and 
displayed  to  the  guard  for  action.  When  the  Savi  RFID  interrogator  was  found  to  be  more  appropriate 
for  MDARS  needs  than  the  Amtech  system,  NOSC  was  further  tasked  by  PSEMO  to  develop  the 
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Product  Assessment  System  (PAS)  extension  shown  within  the  dashed  lines  in  Figure  165  (Smurlo  et 
al.,  1995). 


Supervisor 


Figure  165.  The  NOSC  Multiple  Resource  Host  Architecture  (MRHA)  for  the  MDARS-Interior 
program  is  thought  to  be  the  first  command  and  control  architecture  able  to  oversee  and  direct 
multiple  unmanned  systems  (Laird  et  al.,  1993).  Later  advances  in  multitasking  and  processor 
performance  allowed  the  MRHA  to  run  on  a  single  computer. 

From  the  outset,  Commander  Everett  had  petitioned  PSEMO  to  let  NOSC  develop  the  MRHA  so 
the  government  could  retain  full  rights  (Everett,  1997): 

“In  summary,  reducing  the  scope  of  the  contract  to  only  that  element  the 
government  truly  needs  increases  competition,  simplifies  the  REP  process, 
simplifies  the  source  selection  process,  maximizes  effective  utilization  of  limited 
funds,  seriously  reduces  technical  and  programmatic  risk,  and  saves  the  tax 
payer’s  dollars.” 

In  light  of  the  long-established  DoD  policy  of  picking  a  lead  systems  integrator  from  the  industrial 
sector,  such  an  unorthodox  approach  was  not  an  easy  sell,  but  in  more  recent  times,  the  pendulum  has 
clearly  swung  the  other  way.  The  Weapon  Systems  Acquisition  Reform  Act  of  2009  led  the  Service 
branches  to  rely  more  heavily  on  federally  funded  research,  development,  and  engineering  centers  for 
the  integration  of  mature  technology  solutions.  Most  program  managers  today  insist  on  a 
Government-rights  open  architecture,  an  acquisition  strategy  pioneered  some  25  years  ago  by  the 
MDARS  program. 
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At  the  time,  the  Multiple  Resource  Host  Architecture  represented  an  optimal  command-and-control 
solution  in  terms  of  a  minimal  hardware  configuration  without  significant  performance  tradeoffs. 
High-level  status  (for  all  robots)  was  presented  via  the  Supervisor  display  (Figure  166),  while 
detailed  operational/diagnostic  information  (for  a  selected  robot)  appeared  on  the  Operator  Station 
(Figure  167).  The  initial  MRHA  prototype  was  configured  with  one  Supervisor  Station,  one  Operator 
Station,  two  Planners,  and  one  Link  Server  for  coordinated  control  of  up  to  32  robots  (Laird,  Everett, 
&  Gilbreath,  1993;  Gage  &  Hower,  1994). 41  The  Product  Assessment  System  was  added  to  the 
baseline  configuration  in  1995  (Inderieden,  Everett,  Heath-Pastore,  &  Smurlo,  1995). 


Figure  166.  The  Supervisor  computer  automatically  displayed  the  four  most  relevant  robots,42  which 
were  tracked  in  real  time  on  their  respective  floorplan  maps:  1)  Building  F-36  at  NOSC,  2)  Camp 
Elliott  warehouse  (middle  bay),  3)  Camp  Elliott  warehouse  (south  bay),  and  4)  Cybermotion  facility 
in  Roanoke,  VA. 

The  significance  of  any  "exceptional  events"  sensed  by  the  patrolling  robots  was  assessed  by  the 
MRHA,  based  upon  robot  sensor  and  status  information,  with  operator  assistance  invoked  as  needed 
via  automatic  assignment  to  an  Operator  Station  (Figure  167).  When  the  security-assessment 
software  indicated  a  valid  threat  condition  existed,  an  appropriate  response  was  initiated  by  the 
guard- force  watchstander.  Nonthreatening  events,  on  the  other  hand,  were  handled  with  minimal  if 
any  human  involvement  (e.g.,  autonomously  navigate  around  an  obstruction). 

A  Network  Enabled  Resource  Device  (NERD)  installed  in  a  manned  response  vehicle  investigating 
a  perceived  threat  allowed  the  dispatcher  to  follow  the  response  team’s  progress  on  the  Operator 
Station  (Carroll,  Everett,  Gilbreath,  &  Mullens,  2002).  The  dispatcher  could  also  hand  off  control  to 
a  remote  Operator  Station  situated  inside  the  response  vehicle,  allowing  the  on-scene  team  to  view 
video  from  and  locally  control  the  MDARS  robot  in  cooperative  fashion  (Carroll,  Gilbreath,  Grant,  & 
Day,  2003). 


41  The  ability  for  the  MRHA  to  also  control  outdoor  robots  was  incorporated  under  the  MDARS-Exterior  program  in 
1993  (Heath-Pastore  &  Everett,  1994). 

42  The  security  guard  could  also  zoom  full  screen  on  any  one  map  by  clicking  the  Single  Map  button  at  upper  left. 
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Figure  167.  An  Operator  Station  could  be  automatically  assigned  to  any  specific  robot  when  the 
detailed  attention  of  a  guard  was  required,  allowing  the  supervisor  to  allocate  both  computational 
and  human  resources  in  response  to  exceptional  events  (Everett,  Heath-Pastore,  &  Hower,  1994). 

MDARS  in  all  probability  was  the  first  supervised  autonomous  robotic  installation  to  run  24/7  for 
extended  periods  of  time.  Simultaneous  control  of  two  MDARS  robots  patrolling  nightly  within  an 
unstructured  warehouse  environment  was  demonstrated  for  more  than  2  years  at  a  beta-test  facility 
installed  in  NOSC’s  Camp  Elliott  warehouse  in  San  Diego,  CA  (Figure  168a).  To  establish  a 
navigational  baseline  in  a  structured  (hallway)  environment  for  comparison  with  warehouse 
performance,  an  improved  second-generation  prototype  was  simultaneously  tested  in  the  basement  of 
Building  A33  Topside  at  NOSC  (Figure  168b).  Cybermotion  soon  introduced  their  comparable 
commercial  product  as  the  Security  Robot  2  (SR2)  shown  in  Figure  168c  (Holland,  Martin,  Smurlo, 

&  Everett,  1995). 


Figure  168.  a)  One  of  two  MDARS  prototypes,  circa  1994,  that  patrolled  the  Camp  Elliott  warehouse 
during  an  extended  evaluation,  b)  To  establish  a  navigational  baseline  and  identify  potential  failure 
modes,  a  third  robot  patrolled  nightly  in  Building  A33  at  NOSC,  circa  1996.  c)  The  comparable 
Cybermotion  first-generation  SR2  commercial  version  had  a  video  pan-and-tilt  unit  mounted  beneath 
the  SPI-01  security  payload  (top  center). 
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In  September  1994,  the  Cybermotion  portion  of  the  CRDA  transitioned  to  an  Army  Broad  Agency 
Announcement  (BAA)  contract  to  develop  an  improved  intruder-detection  payload.  The  company’s 
initial  Security  Patrol  Instrumentation  (SPI-01)  payload  was  completely  redesigned  by  Dave  Fisher 
to  include  an  integrated  camera  pan-and-tilt  mechanism.  The  cylindrical  lower  housing  shown  in 
Figure  169a  enclosed  the  tilt  axis  and  rotated  with  the  pan  axis  to  mask  camera  movement  that  would 
otherwise  interfere  with  the  Doppler  motion  detector.  On  the  commercial  side,  Cybermotion 
subsequently  introduced  their  second-generation  SR2  configuration  with  the  SPI-02  payload  (Figure 
169b). 


Figure  169.  a)  The  improved  Cybermotion  SPI-02  and  the  Savi  RFID  interrogator  payloads  during 
testing  in  the  NOSC  Camp  Elliott  warehouse,  circa  1995.  b)  The  comparable  second-generation 
Cybermotion  SR2  CyberGuard  commercial  robot  with  the  SPI-02  security  payload. 

The  second  round  of  Technical  Feasibility  Testing  (TFT-II)  was  conducted  at  Camp  Elliott  in  San 
Diego  in  February  1997  (TFT,  1997).  Since  the  new  SPI-02  payload  reflected  years  of  concept 
development  and  optimization  (under  ROBART II,  the  NOSC/Cybermotion  CRADA,  and  the  BAA 
contract),  the  forthcoming  security-assessment  evaluation  was  viewed  as  relatively  low  risk.  The  far 
bigger  challenge  was  autonomous  navigation  in  a  working  warehouse  environment  (Everett  et  al., 
1984).  The  wide-angle  Cybermotion  sonars  were  good  at  detecting  obstacles  in  front  of  the  robot,  but 
often  lacked  sufficient  angular  resolution  to  find  a  clear  path  in  cluttered  surroundings  as  shown  in 
Figure  170  (Everett,  1995b). 
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Figure  170.  Exhaustive  collision-avoidance  scenarios  were  tested  during  TFT-II  to  ensure  the 
Cybermotion  and  NOSC  sonar  arrays  provided  proper  detection  and  circumnavigation  of  all 
obstacles.  The  yellow  sensors  detected  retroreflective-tape  markers  on  support  columns  along  the 
aisle  for  improved  localization  (Borenstein,  Everett,  and  Feng,  1996). 43 

Originally  developed  on  ROBART II,  the  complementary  narrow-beam  sonar  array  did  a  much 
better  job  identifying  clear  pathways  the  robot  could  traverse  (Everett,  1995b).  As  shown  in  Figure 
171,  for  example,  range  and  bearing  data  from  the  Polaroid  sonar  array,  mounted  under  the 
Navmaster  turret  above  and  behind  the  tactile  bumper,  caused  the  robot  to  deflect  to  the  left  of  the 
stepladder  for  safe  passage  through  a  narrow  pathway.  Without  this  higher  resolution  narrow-beam 
sonar  input,  the  collision-avoidance  software  would  have  concluded  the  aisle  was  completely 
blocked,  preventing  the  robot  from  continuing  its  assigned  patrol  segment. 


Figure  171 .  a)  The  robot  properly  concluded  it  could  not  go  through  the  perceived  opening  under  the 
ladder  and  altered  course  accordingly  to  the  left,  b)  The  MRHA  command-and-control  console  was 
set  up  inside  an  air-conditioned  container  van  outside  the  Camp  Elliot  warehouse  for  continuous 
24/7  testing. 


43  U.S.  Patent  No.  5,812,267  was  awarded  to  H.  R.  Everett,  G.A.  Gilbreath,  R.S.  Inderieden,  T.T  Tran,  and  J.H. 
Holland  for  this  optical  localization  concept  on  22  September,  1998 
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TFT-II  testing  at  Camp  Elliot  was  initiated,  monitored,  and  recorded  from  the  MRHA  console  in  an 
outdoor  air-conditioned  container  van  (Figure  171b),  with  PSEMO-designated  observers  taking  notes 
inside  the  warehouse  (Figure  172a).  The  robot  successfully  executed  each  collision-avoidance  test 
scenario  with  no  human  intervention,  thereby  achieving  a  perfect  score  for  this  event.  In  addition, 
extensive  tests  were  conducted  to  evaluate  the  Intrusion  Detection  System  and  Product  Assessment 
payloads  (Figure  172b),  with  specific  metrics  collected  on  false  alarm  rate,  nuisance  alarm  rate, 
probability  of  detection,  and  product  status  verification  (TFT,  1997). 


Figure  172.  a)  Representatives  of  the  MDARS  team  in  the  Camp  Elliott  Warehouse  during  TFT-II 
(left  to  right):  Brendan  Smith,  John  Bott,  Dori  Jaffee,  Steve  Timmer,  Tracy  Pastore,  Doug  Murphy, 
Jerry  Edwards,  robot,  Larry  Drymon,  Lieutenant  Colonel  Bruce  Swagler,  Gary  Gilbreath,  “Ish,”  Bart 
Everett,  and  Jeff  Garwood,  b)  The  Intrusion  Detection  System  ( SPI-02 )  and  Product  Assessment 
(RFID  interrogator)  payloads  were  tested  on  warehouse  patrols. 

In  1997,  a  pair  of  MDARS  Interior  robots  was  installed  within  the  Defense  Logistics  Agency 
(DLA)  warehouse  at  Anniston  Army  Depot,  AL  (Newsome,  1998).  Considerable  effort  was  spent 
debugging  this  installation  at  Anniston  while  still  developing  code  on  the  fly  (Figure  173),  as  the 
Army  needed  to  expedite  the  schedule  to  secure  continued  funding.  Bart  Everett  divided  his  team 
into  two  groups  that  alternately  travelled  every  other  week,  which  he  supervised  on  each  trip.  These 
teams  worked  on  the  warehouse  installation  from  0600  to  1530  when  the  local  workers  locked  up, 
wrote  and  debugged  software  in  the  adjacent  MDARS  trailer  until  1730,  then  relocated  to  the  Guard 
Shack  for  several  more  hours  on  the  control  console. 
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Figure  173.  Electronics  technician  Steve  Timmer  of  Code  D371  worked  long  hours  on  multiple  bi¬ 
weekly  trips  to  Anniston  installing  and  maintaining  the  two  MDARS-Interior  robots  in  the  DLA 
warehouse.  Invaluable  assistance  was  also  provided  by  John  Holland  and  his  engineering  support 
team  from  Cybermotion. 

Early  User  Appraisal  (EUA)  of  the  MDARS-Interior  program  began  in  November  1997,  during 
which  the  two  robots  patrolled  the  warehouse  for  another  8  months  (Newton,  1998).  The  security 
guards  of  the  Anniston  Army  Depot  police  force  each  received  2  hours  of  operator  training  from 
Robin  Laird  (Figure  174a),  which  heralded  the  first  time  the  MDARS-Interior  system  was  in 
operational  mode  under  actual  user  control  (Figure  174b).  Some  14  demonstrations  were  held  for 
various  DoD  and  DLA  officials  during  EUA,  all  successful,  with  the  final  event  hosting  some  75 
members  of  the  Physical  Security  Equipment  Action  Group. 


Figure  174.  a)  Robin  Laird  prepared  and  individually  presented  the  MDARS  operator  training 
syllabus  to  the  guard  force  at  Anniston  Army  Depot,  b)  The  local  guard  force  then  took  over  control 
of  the  MDARS  robots  in  the  remote  DLA  warehouse  for  the  duration  of  Early  User  Appraisal. 
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In  April  1999,  the  Army  Program  Office  awarded  an  Engineering  Manufacturing  Development 
(EMD)  contract  to  General  Dynamics  Robotic  Systems  (GDRS),  formerly  Robotic  Systems 
Technology  (RST),  Westminster,  MD.  The  GDRS  proposal  had  included  the  Cybermotion  K2A 
platform  with  the  SPI-02  security  payload,  which  had  been  extensively  tested  during  EUA  in  the 
DLA  warehouse  at  Anniston  Army  Depot.  When  Cybermotion  replaced  their  K2A  platform  with  the 
newer  K3A  version,  however,  GDRS  amended  its  contract  to  delete  the  K2A  platform  and  specify  this 
newer  model. 

The  resulting  MDARS-Interior  System  Development  and  Demonstration  (SDD)  platform  (Figure 
175a)  was  based  upon  a  K3A  SPI-Master  outfitted  with  MDARS-spec'ific  subsystems  in  support  of 
two  primary  missions:  1)  patrol  a  pre-specified  area  and  check  for  intruders,  and  2)  read  RFID  tags 
affixed  to  sensitive  or  high-value  inventory.  Mission  modules  included  an  intrusion-detection 
payload  that  fused  microwave  and  passive-infrared  motion  sensors,  a  video  camera  for  surveillance, 
and  a  Savi  tag  reader  for  inventory  assessment.  Obstacle  avoidance  was  supported  with  ultrasonic 
sensors,  two  scanning  lidars,  and  a  tactile  bumper. 


a) 


b) 


Figure  175.  a)  The  MDARS-Interior  SDD  system,  based  on  the  K3A  platform  and  the  SPI-02 
security-sensor  payload,  underwent  Limited  User  Testing  (LUT)  at  Susquehanna  Army  Depot,  PA. 
b)  The  corresponding  Cybermotion  Cyberguard  SR3  commercial  product  consisted  of  its  K3A 
platform  equipped  with  the  SPI-01  payload. 

In  February  2001,  the  MDARS-Interior  system  began  LUT  at  Susquehanna  Army  Depot,  PA 
(Everett,  2003).  This  evolution  evaluated  the  same  system  functionality  addressed  at  the  Early  User 
Appraisal  at  Anniston  Army  Depot  in  1997,  but  with  a  greater  emphasis  on  product  assessment.  In 
2003,  however,  Cybermotion  unfortunately  discontinued  operations  and  disbanded,44  whereupon  the 
Army  Program  Office  placed  the  MDARS-Interior  program  in  abeyance,  as  user  interest  had  shifted 
more  towards  exterior  applications.  The  development  history  of  the  interior  security-robot  concept 
over  a  22-year  timeframe  is  graphically  depicted  in  the  poster  reproduced  as  Figure  176. 


44  Cybermotion’s  patented  Synchro-Drive  transmission  provided  extremely  accurate  dead-reckoning  capability  to 
their  omni-directional  mobility  base,  but  was  rather  expensive  to  produce.  The  advent  of  laser-based  localization 
schemes  allowed  far  less  precise  mobility  solutions  to  be  used,  resulting  in  lower-cost  commercial  alternatives. 
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Figure  176.  The  evolution  of  the  interior  security  robot  began  with  ROBART I  in  late  1980  and 
progressed  through  several  phases  of  increased  sophistication  over  a  span  of  22  years.  While  DoD 
never  fielded  such  robots  indoors,  the  underlying  technology  was  repurposed  for  far  greater  return 
on  investment  under  the  MDARS-Exterior  project. 
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ROB  ART  ///(1 992-2007) 


Like  its  predecessors,  ROB  ART  III  was  a  laboratory  surrogate,  never  intended  for  real-world 
operation  (Figure  177):  1)  it  was  not  waterproof;  2)  its  mobility  was  constrained  to  planar  surfaces, 
so  it  could  not  ascend  or  descend  stairs;  3)  it  was  not  defensively  armored;  4)  it  was  not  rugged;  and 
5)  it  could  not  right  itself  in  the  event  it  flipped  over.  Instead,  ROBART  III  was  a  concept/software- 
development  platform  built  in  Bart  Everett’s  garage,  optimally  configured  to  support  a  development 
role  in  a  laboratory  environment.  The  continuously  evolving  prototype  made  many  contributions  to 
the  field  of  both  supervised  and  fully  autonomous  robots,  to  include  perception,  localization, 
navigation,  and  response  (Ciccimaro  &  West,  1998). 


Figure  177.  a)  This  1992  photo  of  ROBART  III  shows  the  6502-based  video  motion  detector 
developed  under  ROBART  II  atop  the  head,  with  its  pin-hole  camera  midway  between  the  two  head- 
mounted  sonar  transducers,  b)  The  central  card  cage  has  been  vertically  extended  to  accommodate 
more  single-board  computers,  circa  1995. 

Numerous  hardware  upgrades  were  made  during  ROBART  Ill’s  15-year  lifetime  in  support  of  more 
sophisticated  navigation,  collision  avoidance,  and  mapping  schemes,  to  include  a  MicroStrain  gyro- 
stabilized  magnetic  compass,  a  KVH  fiber-optic  rate  gyro,  and  a  2D  Sick  LMS-200  scanning  lidar 
(Figure  178).  Full-duplex  data  communication  with  the  PC -based  control  station  was  accomplished 
via  a  9600-baud  Telesystems  spread-spectrum  RF  link.  The  12-volt  ABEC  drive  motors  were 
identical  to  those  used  on  ROBART II,  but  equipped  with  higher  traction  snow  tires  (Figure  177b). 
System  power  was  supplied  by  an  80-amphour  12-volt  gel-cell  battery  that  provided  for  several  hours 
of  continuous  operation  between  charges. 
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Figure  178.  a)  Early  configuration  of  ROBART  III  showing  the  five-element  sonar  array  on  the 
mobility  base,  circa  2000.  b)  The  five-element  array  has  been  replaced  with  a  four-element  array  and 
a  2D  Sick  lidar  incorporated  into  a  front  bumper,  circa  2002.  Note  upgraded  LED  headlights  in  the 
black  heat  sinks  above  the  sonar  transducers. 


Referring  again  to  Figure  178a,  early  versions  of  ROBART II  had  a  temporary  black/yellow  pigtail 
hanging  from  the  mobility  base  for  manual  battery  charging.  While  the  front  bumper  design  shown  in 
Figure  178b  was  primarily  added  in  2002  to  support  the  Sick  lidar,  it  also  accommodated  automatic 
recharging  and  tactile  sensing  as  shown  in  Figure  179.  For  compatibility  with  existing  battery 
chargers  for  ROBART  I  and  ROBART  II,  the  1-inch  aluminum  contact  strip  on  the  front  tactile- 
bumper  segment  (image  top  center)  served  as  the  negative  recharging  contact.  Similarly,  the 
descending  vertical-spring  contact  at  image  center  enabled  the  positive  connection. 


Figure  179.  Bottom  view  of  the  tactile  bumper  and  Sick  lidar  support  assembly,  showing  the  four 
black  microswitches  that  sensed  bumper-segment  contact,  the  “negative”  battery-charging  contact 
on  the  front  bumper  segment  (top  center),  and  the  vertical-spring  “positive”  charging  contact 
(center).  See  also  Figure  178b. 
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In  the  mid-2000s,  three  16-bit  computers  were  added  to  the  on-board  architecture  to  support  more 
advanced  autonomy  (Figure  180):  1)  the  torso  computer  processed  sonar  range  data,  speech  output, 
and  integrated  motion  of  the  surrogate  weapon  and  head,  2)  the  vision  computer  in  the  head  was 
processed  live  video  from  the  omnidirectional  and  pan-and-tilt  cameras  and  3)  the  drive  computer  in 
the  mobility  base  controlled  the  drive  motors  in  response  to  data  from  the  torso  computer,  Sick  lidar, 
KVH  fiber-optic  gyro,  and  the  Micro  Strain  compass.  Multiple  8-bit  microcontrollers  were  still 
employed  for  low-level  sensor  processing  and  actuator  control. 


Figure  180.  Electronics  technician  Yuong  Sun  (left)  and  electrical  engineer  Brandon  Sights  (right) 
reassemble  ROBART ///following  extensive  reconfiguration  to  accommodate  the  new  16-bit  torso 
computer  in  2006. 

Intended  for  concept-development  and  demonstration,  the  non-lethal-response  weapon  shown  in 
Figure  181  was  a  pneumatically  powered  dart  gun  capable  of  firing  a  variety  of  3/16-inch  diameter 
projectiles  (Ciccimaro,  Everett,  Gilbreath,  &  Tran  1998).  By  way  of  example,  simulated  tranquilizer 
darts  (20-gauge  spring-steel  wires  terminated  with  3/1 6-inch  nylon  balls)  illustrated  a  potential 
response  application  involving  remote  firing  of  incapacitating  rounds  by  military  or  law  enforcement 
personnel.  A  rotating-barrel  arrangement  was  incorporated  to  allow  for  multiple  firings  (six)  with 
minimal  mechanical  complexity.  The  spinning  barrel  also  imparted  a  rather  sobering  psychological 
message  during  system  initialization. 
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Figure  181 .  A  secondary  compressed-air  accumulator  (bottom  center)  mounted  on  the  pneumatic 
non-lethal-weapon  prototype  was  maintained  at  a  constant  pressure  of  120  psi,  replenished 
automatically  from  the  larger  150-psi  accumulator  (far  left).  Note  Micro  Strain  compass  on  fiberglass 
pedestal  at  lower  left. 

The  darts  (or  steel  balls)  were  fired  by  a  release  of  compressed  air  stored  in  a  pressurized 
accumulator  at  the  rear  of  the  gun  assembly.  This  accumulator  was  monitored  by  a  Micro  Switch 
242PC150G  electronic  pressure  transducer  and  maintained  at  a  constant  pressure  of  120  psi  by  a 
second  solenoid  valve  (Figure  182).  To  minimize  air  loss,  the  solenoid  valve  linking  the  gun 
accumulator  to  the  active  barrel  was  opened  for  precisely  the  amount  of  time  required  to  expel  the 
projectile.  All  six  darts  could  be  fired  in  approximately  1.5  seconds  under  repeatable  launch 
conditions  to  ensure  accurate  performance.  A  visible-red  laser  gunsight  was  provided  to  facilitate 
manual  as  well  as  automatic  targeting. 


Figure  182.  The  black  electronic  pressure  transducer  for  the  gun  accumulator  is  on  the  starboard 
bulkhead  (upper  left),  with  the  bank  of  yellow  solenoid  valves  just  to  the  right  on  the  transverse 
bulkhead  inside  the  drive-motor  compartment  of  the  mobility  base.  The  battery  compartment  is  in  the 
background. 
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The  surrogate  weapon  was  designed  for  laboratory  use  only,  supporting  vision-based  weapon 
control  without  undue  risk  to  personnel.  A  fiber-optic  sensor  on  the  gun  determined  load  status  for 
each  barrel  (Figure  183a).  A  local  Ready/Standby  switch  enabled  the  air  compressor  and  secondary 
accumulator  charging,  and  a  local  Arm/Safe  switch  physically  interrupted  power  to  the  trigger 
solenoid  (Figure  183b).  There  were  parallel  software  disables  for  both  these  same  functions  on  the 
remote  OCU.  Two  separate  control  lines  were  employed  for  the  trigger  solenoid,  one  active-high  and 
the  other  active-low,  ANDed  together  to  minimize  inadvertent  activation  during  initialization  or  in 
the  event  of  a  computer  reset.  Redundant  emergency  overrides  were  also  provided:  1)  two  local 
E-Stop  buttons  on  the  mobility  base,  2)  an  RF-kill  pendent,  and  3)  a  remote  E-Stop  button  at  the 
control  station.45 


b) 

Figure  183.  a)  The  yellow  proximity  sensor  adjacent  to  the  blue  solenoid  valve  was  fiber-optically 
coupled  to  determine  the  load  status  of  each  barrel  as  it  rotated.  Note  red  E-Stop  button  at  lower  left, 
b)  The  Ready/Standby  switch  (upper  left)  and  Arm/Safe  switch  (upper  right)  above  the  main 
accumulator  locally  controlled  weapon  status. 

Assisted  weapon  control,  an  important  new  behavior  introduced  on  ROB  ART  III,  was  an  extension 
of  the  reflexive-teleoperation  concept  developed  on  ROB  ART  II.  The  issue  of  concern  was  the 
difficulty  encountered  when  teleoperating  a  mobile  robot  equipped  with  surveillance  and/or  targeting 
cameras,  plus  an  articulated  weapon  system.  Experience  gained  through  extended  use  of 
conventional  teleoperated  devices  of  this  type  had  revealed  considerable  shortcomings  from  a 
man-machine  interface  point  of  view.  Simply  put,  if  a  remote  operator  has  to  master  simultaneous 
manipulation  of  three  different  joysticks  (i.e.,  one  for  drive  and  steering,  another  for  camera  pan  and 
tilt,  and  yet  a  third  for  weapon  control),  the  chances  of  hitting  a  moving  target  are  minimal. 

Our  initial  approach  in  addressing  this  problem  involved  making  two  of  the  three  controllable 
elements  (i.e.,  drive  control,  camera  control,  and  weapon  control)  slaves  to  the  third,  so  the  human 
operator  only  had  to  deal  with  one  entity  (Figure  184).  For  example,  the  head-mounted  surveillance 
camera  could  be  slaved  to  the  weapon  so  the  camera  looked  wherever  the  operator  pointed  the  gun.  If 
either  the  weapon  pan-axis  controller  or  the  camera  pan-axis  controller  approached  their  respective 
limits  of  travel,  the  mobility  base  automatically  rotated  in  place  or  turned  in  the  proper  direction  to 


45  Activation  of  any  E-Stop  input  device  immediately  disabled  both  the  weapon  and  mobility  systems. 
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restore  the  necessary  range  of  payload  motion.  Alternatively,  the  weapon  could  be  slaved  to  the 
surveillance  camera,  and  so  forth. 


Figure  184.  a)  The  Navigation  Control  Screen  for  ROBART  III.  b)  The  “Slave”  window  on  the 
Weapon  Control  Screen  (upper  left  quadrant)  allowed  the  head  pan  axis  and/or  the  mobility  base  to 
be  slaved  to  respond  to  pan-axis  deflection  of  the  surrogate  mini-gun  (adapted  from  Gilbreath, 
Ciccimaro,  &  Everett,  2000). 

Taking  things  a  step  further,  final  closed- loop  control  of  weapon  pan-and-tilt  could  be  provided  by 
the  video  target-acquisition  system.  Initial  360-degree  motion  detection  for  this  behavior  was 
supported  by  a  ring  of  passive  infrared  sensors  around  the  neck,  an  AM  Sensors  microwave  motion 
detector  behind  the  faceplate,  and  a  Visual  Stone  omni-directional  camera  mounted  on  the  head 
(Figure  185).  Fused  outputs  from  these  sensors  were  used  to  cue  a  Canon  high-resolution  pan-tilt- 
zoom  (PTZ)  camera  in  azimuth  and  elevation,  which  provided  video  of  potential  targets  for  further 
assessment  and  classification.  Final  closed-loop  weapon  control  was  provided  by  gun-camera 
imagery. 


Figure  185.  The  head-mounted  high-resolution  PTZ  camera  was  cued  to  the  location  of  any 
suspected  disturbance  sensed  by  the  360-degree  motion-detection  sensors  (eight  passive-infrared 
detectors  around  the  neck,  the  Visual  Stone  omni-directional  camera  at  top  center,  and  a  microwave 
motion  detector  behind  the  face  plate. 
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Our  chosen  method  of  object  classification  inherently  possessed  a  large  degree  of  modularity.  If 
multiple  objects  were  to  be  detected,  then  each  cascade  of  weak  classifiers  could  simply  be  swapped 
out  with  a  new  set  that  had  been  trained  to  detect  the  current  object  of  interest.  The  detection 
efficiency  of  a  trained  set  of  boosted  classifiers  was  sufficient  to  pass  control  to  another  more 
discriminating  detection  mechanism  without  compromising  real-time  performance.  Some  possible 
second-stage  detection  mechanisms  included  using  the  bore-sighted  laser  to  lock  on  and  fire  at  a 
confirmed  target,  or  utilizing  a  text-parser  to  interpret  letters  on  a  license  plate  or  sign  (Chen  & 
Yuille,  2004). 

The  license-plate  detection  method  of  Dlagnekov  (2005)  was  investigated  to  evaluate  the  Adaboost 
machine-leaming/training  algorithm  for  generic  object  detection  (Figure  186),  using  ordinary  soda 
cans  to  quantify  performance  with  respect  to  scale,  background  clutter,  and  changes  in  specularity. 

To  build  a  training  set,  the  cans  were  randomly  emplaced  in  a  cluttered  lab  area  at  a  variety  of 
distances  and  orientations  (Figure  186a).  Footage  from  ROBART IIFs  head-mounted  camera  was 
captured  under  three  different  lighting  configurations  as  three  separate  MPEG  videos,  at  a  pixel 
resolution  of  320  x  240  with  a  frame-rate  of  3  frames-per-second  (Kogut  et  al.,  2006). 


Figure  186.  a)  Example  of  an  extracted  video  frame  where  each  actual  soda-can  region  has  been 
manually  labeled,  b)  Example  54  x  83-scale  image  showing  both  positive  and  negative  soda-can 
detections. 

Each  video  was  divided  into  two  equal  segments,  with  a  portion  from  each  segment  pair  randomly 
selected,  resized  to  a  resolution  of  720  x  480,  and  extracted  as  a  sequence  of  bitmap  images,  which 
were  set  aside  to  build  the  training  set.  Soda-can  images  were  manually  labeled  in  every  8th  frame  of 
the  bitmap  training  images,  then  sorted  into  four  separate  groups  based  on  their  image  dimensions 
(Figure  187a).  The  remaining  portions  were  merged  together  and  used  as  test  footage  for  the 
detection  algorithm.  Despite  a  few  anomalies,  the  detection  windows  shown  in  Figure  187b  allowed 
ROBART  III  to  effectively  aim  the  pneumatically  powered  weapon  surrogate,  albeit  under  ideal 
laboratory  conditions  (Kogut  et  al.,  2006). 
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Figure  187.  a)  A  composite  representation  showing  four  different  scales  of  acquired  training  images, 
where  the  black  areas  in  the  bottom-right  regions  indicate  no  soda-can  images  were  present 
(adapted  from  Kogut  et  al.,  2006).  b)  Real-time  can  detections  using  recorded  footage  from 
ROBART  Ill’s  camera  (adapted  from  Kogut  et  al.,  2006). 

The  robot’s  PTZ  camera  protocol  was  later  integrated  with  a  two-stage  search-and-engage 
algorithm  that  performed  a  wide-area  scan  for  a  pre-taught  class  of  objects,  then  zoomed  in  to  look 
for  specific  “vulnerabilities”  assigned  to  that  particular  target.  For  the  scenario  depicted  in  Figure 
188,  the  sought  object  was  the  dark  wooden  box  atop  a  mobile  pedestal  shown  earlier  in  Figure  187b, 
for  which  the  designated  “vulnerability”  was  a  soda  can  inside  the  box.  The  surrogate  weapon  was 
automatically  trained  accordingly,  using  the  gun-camera  video  and  bore-sighted  targeting  laser,  then 
fired  under  operator  supervision. 


Figure  188.  Viewing  sequential  gun-camera  video  frames  from  left  to  right:  a)  the  laser-sight  icon  is 
superimposed  upon  the  initially  detected  can  position;  b)  the  can  is  then  moved  to  a  new  location 
and  visually  tracked  en  route;  c)  the  laser-sight  icon  begins  to  pursue  the  can;  d)  the  laser-sight  icon 
comes  to  rest  with  the  can  again  targeted. 

Detection  of  the  laser  spot  was  disambiguated  by  grabbing  two  successive  frames  of  video  with  the 
laser  sight  both  on  (Figure  189a)  and  off  (Figure  189b),  then  subtracting  the  two  images  to  yield  a 
binary  difference  image.  This  process  continued  as  the  Weapon  Controller  sent  real-time  error-based 
pan-and-tilt  velocity  commands  to  reposition  the  laser  spot  on  target.  Note  spring-steel  darts 
terminated  by  plastic  balls  embedded  in  the  target.46  Results  of  this  successful  feasibility 
demonstration  were  transferred  to  the  Man-Portable  Perimeter  Protection  (MP3)  and  Networked 
Remotely  Operated  Weapon  System  (NROWS)  projects  discussed  in  later  sections. 


46  Although  Figure  189  shows  darts,  we  typically  fired  3/16-inch  steel  balls  at  the  defenseless  soda  cans. 
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Figure  189.  a)  The  spot  of  illumination  shows  up  in  the  video  image  when  the  laser  sight  is  on. 
b)  Here  the  laser  sight  has  been  turned  off,  so  the  two  images  can  be  subtracted  to  yield  the  spot 
location.  The  Weapon  Controller  calculated  proportional  pan-and-tilt  commands  to  progressively 
move  the  detected  laser  spot  onto  the  bulls-eye. 

In  the  summer  of  2006,  a  two-rail  surrogate-missile  payload  that  fired  rubber-tipped  8.5-inch 
plastic  missiles  was  added  to  the  left  shoulder  pod  (Figure  190a).  Sold  as  part  of  a  toy  rocket 
launcher,  these  missiles  could  attain  an  altitude  of 400  feet,  powered  by  a  child  stomping  on  a  rubber 
air  bag  connected  to  the  vertical  launch  tube.  The  addition  of  this  payload  required  a  larger  main 
accumulator,  and  a  higher-throughput  mechanical  regulator  to  maintain  the  secondary  accumulator  at 
a  constant  pressure  of  120  psi  (Figure  190b).  When  fired  at  a  45-degree  angle,  these  surrogate 
missiles  could  easily  travel  about  half  a  block  downrange. 


a) 


b) 


Figure  190.  a)  The  surrogate  missile  launcher  added  to  the  left  shoulder  mount  in  2006  consisted  of 
two  plastic  rockets  on  1/2-inch  tubular  rails,  coupled  via  quick  acting  two-stage  solenoid  valves  to  a 
local  accumulator  immediately  above,  b)  A  larger  main  accumulator  and  mechanical  pressure 
regulator  were  added  in  2007. 
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Rather  than  relying  upon  a  large  catch-all  object-detection  approach,  Code  2371  implemented 
numerous  specialized  detection  methods  that  could  be  selectively  applied  once  the  robot  had 
established  suitable  context  for  their  deployment  (Kogut  et  al.,  2006).  While  the  traditional  approach 
often  views  computer  vision  as  a  more  isolated  task,  our  vision  development  took  full  advantage  of 
both  the  robot’s  navigational  and  perception  capabilities  to  simplify  the  procedure.  Figure  191  shows 
the  results  of  a  doorway-detection  algorithm  that  analyzed  the  SLAM  data  collected  by  the  Sick  lidar 
during  building  exploration,  with  doorway  icons  inserted  into  the  map  representation  to  flag  open 
doorways  that  subsequently  cued  the  vision  system  to  look  for  room  signs. 


Figure  191 .  Created  in  April  2008,  this  augmented  virtuality  map  of  Buildings  622  (right)  and  624 
(left)  shows  doorway  icons  (black  rectangles  inside  brown  frames)  inserted  into  the  SLAM  map 
representation  to  mark  the  perceived  location  of  open  doorways  as  detected  by  the  2D  Sick  lidar. 

Rather  than  continuously  search  the  full  field  of  regard  for  signs  from  which  to  extract 
information,  the  robot  instead  optimally  positioned  itself  to  investigate  the  much  smaller  wall  surface 
to  either  side  of  any  detected  doorway,  using  the  previously  discussed  boosted  classifier  to  look  for  a 
rectangular  shape  (Kogut  et  al.,  2006).  If  a  suitable  shape  was  detected,  the  robot  would  next  verify 
there  were  characters  within  this  shape,  then  interpret  the  sign  using  optical  character  recognition 
(Figure  192).  Under  this  heuristic  task-decomposition  approach,  conditions  that  significantly 
enhanced  the  performance  of  visual  detection  were  optimally  provided  by  the  robot,  thereby  enabling 
more  intelligent  autonomous  behavior. 


File  Edit  View  Terminal  Tabs  Help 


mck@squarepants :  ~/sign_detect 


ni  ck@squ  a  repant  s :  ~/°pt_char_recog$  .  /driver 
104 

Estrellina  Pacis 
Electrical 

Adaptive  Systems  Branch 
Code  2371 


m ek@squ a repant s : ~/°pt_char_recog$  | 


Figure  192.  The  above  information  regarding  the  office  room  number,  occupant,  and  organizational 
code  was  captured  by  the  video  camera,  which  allowed  the  robot  to  subsequently  respond  to  voice 
or  e-mail  commands  such  as  “go  to  room  104,”  or  “go  to  Estrellina’s  office.”  See  Figure  191. 
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A  natural-language  interface  allowed  ROBART III  to  receive  fairly  unstructured  verbal  direction, 
no  different  from  the  procedures  used  to  instruct  a  human  to  perform  the  same  task.  For  example, 
suppose  the  robot  had  penetrated  an  underground  bunker  and  was  streaming  back  video  that  showed 
an  open  doorway  in  the  center  of  the  far  wall.  A  human  monitoring  this  video  might  converse  with 
the  robot  as  follows:  “Enter  the  doorway  in  front  of  you.”  The  robot  would  then  look  for  predefined 
scene  attributes  that  suggested  a  door  frame  or  opening,  highlighting  its  choice  with  a  graphic 
overlay,  whereupon  the  human  could  confirm  or  redirect  as  needed. 


Figure  193.  a)  Initial  e-mail  image  request  from  Estrellina  Pads  in  Building  622  to  ROBART  III  in 
Building  624  dated  21  October  2004  @  1 :42  PM.  b)  Subsequent  response  dated  21  October  2004  @ 
1:42  PM. 

To  eliminate  voice-recognition  problems  during  the  initial  stages  of  development,  ROBART  III  was 
assigned  a  working  e-mail  account,  thus  enabling  human-robot  interaction  via  simple  text  strings, 
with  the  added  ability  to  enclose  return  attachments.  Figure  193a  above  shows  an  outgoing  e-mail 
request  from  Estrellina  Pacis  to  ROBART  III  requesting  an  image  dump  from  the  head-mounted  PTZ 
video  camera.  The  automatic  response  shown  in  Figure  193b  returned  a  JPG  frame  grab  of  the 
current  camera  view,  in  this  particular  case  depicting  the  robot’s  approach  to  its  linear  docking 
station  in  an  adjacent  building  (Figure  194a).  The  desired  video  source  and  camera  pose  could  be 
specified  by  e-mail  or  automatically  determined  by  the  behavior  under  execution,  as  in  this  particular 
case  involving  a  recharging  event  (Figure  194b). 


Figure  194.  a)  ROBART  III  approaches  its  linear  docking  station  in  what  is  now  the  museum  area  of 
Building  624.  b)  The  returned  image  shows  the  PTZ  camera  looking  down  at  the  charging  strip.  This 
feature  was  extremely  useful  for  monitoring  performance,  as  well  as  for  retasking  the  robot  from  a 
remote  location. 
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One  of  the  problems  associated  with  adding  autonomy  to  man-portable  robots  employed  in 
counter-IED  missions  is  the  consequent  per-unit  price  tag.  As  even  the  teleoperated  EOD  robots 
provided  by  QinetiQ  and  iRobot  cost  over  $1 10,000  apiece,  efforts  to  provide  cheaper  (and  lighter) 
alternatives  were  soon  explored  by  a  number  of  organizations.  One  option  involved  the  use  of  radio- 
controlled  toys  to  deliver  an  explosive  charge  that  could  be  remotely  detonated  to  neutralize  the  IED. 
The  MARCbot  I  was  developed  by  Exponent  for  the  U.S.  Army  Rapid  Equipping  Force  in  2002,  for 
example  (Figure  195a),  and  the  BomBot  (Figure  195c)  was  prototyped  in  2005  by  the  Air  Force 
Research  Lab  (Vickers,  2005;  AFRL,  2005). 


Figure  195.  a)  Developed  by  Exponent,  the  MARCbot  I  was  deployed  in  Afghanistan  in  July  2002.  b) 
The  improved  MARCbot  IV  (Exponent,  2016).  c)  The  Innovative  Response  Technologies  BomBot 
was  produced  for  NAVEODTECHDIV  under  a  NAVSEA  contract  (Lash,  2006). 

The  basic  idea  involved  remote  delivery  of  a  C4  charge  that  could  be  ejected  onto  a  suspected  IED 
by  a  low-cost  line-of-sight  UGV,  then  remotely  detonated  once  the  R/C  UGV  had  been  withdrawn  by 
its  operator  (USR,  2007).  To  achieve  the  necessary  operator  standoff,  however,  a  video  camera  had 
to  be  added  to  the  system,  which  required  a  frame  grabber,  codec,  and  upgraded  RF  link, 
significantly  increasing  the  cost.  The  initial  $7000  price  tag  of  a  MARCbot  soon  rose  to  $15,000,  and 
the  unit  cost  under  the  NAVSEA  contract  was  reportedly  $45,000  (Stevens,  2006).  To  get  around  this 
problem,  we  introduced  the  concept  of  autonomous  teleoperation,  which  allowed  a  sophisticated 
(expensive)  robot  to  serve  as  a  surrogate  controller  for  the  expendable  R/C  vehicle,  supervised  by  a 
human  operator  even  further  away  (Figure  196). 


IED 


(8)  VS)  . 
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R/C  Link 
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High-Value 
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Figure  196.  The  “autonomous  teleoperation”  concept  involves  a  low-cost  (expendable)  robot  that 
emplaces  a  destruct  charge  on  a  suspected  IED  (left),  controlled  by  an  intelligent  (high-value)  robot 
some  75  meters  away  (center),  which  in  turn  can  be  supervised  by  a  remote  human  operator  500 
meters  further  still  (right). 
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As  demonstrated  on  ROBART III,  the  supervisory  human  even  became  optional,  essentially 
allowing  the  high-value  robot  to  deploy  and  recover  its  own  sacrificial  drone  scout  (Figure  196b).  As 
the  envisioned  application  for  this  upgrade  was  autonomous  structure  exploration,  we  decided  to  use 
a  slave  UAV  versus  a  UGV  to  facilitate  mobility  in  damaged  or  cluttered  environments  (Kogut  et  al., 
2006).  Initial  attempts  involved  a  toy  R/C  helicopter  (Figure  197a),  which  proved  too  unstable,  so  we 
switched  to  an  R/C  blimp  (Figure  197b),  which  although  stable,  was  too  susceptible  to  air  currents  to 
be  practical.  This  video-based  remote-control  software  was  developed  in  a  matter  of  days  by 
electrical  engineer  Greg  Kogut  using  a  single-chip  RS-232  to  R/C  converter. 


Figure  197.  a)  Public  Affairs  Specialist  Ann  Dakis  demonstrates  the  ability  of  a  previously  untrained 
human  to  effectively  fly  an  R/C  helicopter,  b)  Using  a  temporary  R/C  controller  for  the  blimp  taped  to 
the  left-rear  corner  of  its  head,  ROBART  III  visually  guides  a  dirigible  towards  an  open  doorway  in 
Building  624,  circa  April  2006. 

In  addition  to  numerous  other  efforts,  ROBART  III  served  as  a  transition  platform  for  the  JGRE- 
funded  Technology  Transfer  project  managed  by  Estrellina  Pacis  (Figure  198),  which  is  further 
discussed  in  Volume  2.  This  multi-year  endeavor  harvested  newly  developed  component 
technologies  from  various  sources  to  increase  the  functionality  and  autonomy  of  man-portable  robots 
(Newton,  2006).  Key  research  thrusts  included:  1)  enhanced  reflexive  teleoperation,  2)  automated 
target  acquisition  and  tracking,  3)  simultaneous  localization  and  mapping,  4)  natural-language 
understanding,  and  5)  augmented  virtuality. 
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Figure  198.  Tech  Transfer  project  manager  and  electrical  engineer  Estrellina  Pads  (left)  works  with 
mechanical  engineer  Ben  Stratton  (seated  right)  to  modify  ROBART  III  in  support  of  the  an 
upcoming  feasibility  demonstration. 

As  one  of  the  more  sophisticated  mobile-robot  research  platforms  of  its  time,  ROBART  III  was 
featured  numerous  times  on  the  Learning  (Figure  199),  History,  and  Discovery  Channels  (Newton, 
2006),  and  in  January  2006  was  ranked  number  16  in  Wired  Magazine's  survey  of  the  50  best  robots 
ever  (Anderson,  2006).  Dr.  John  Silva  of  NOSC  had  estimated  that  ROBART  //probably  saved  the 
Navy  a  million  dollars  in  development  costs,  and  an  even  greater  savings  can  be  attributed  to 
ROBART  III  { Davis,  201 1),  which  served  as  an  application-development  surrogate  for  15  years.  Both 
systems  significantly  contributed  to  the  Center’s  growing  reputation  as  the  go-to  government 
laboratory  for  intelligent  unmanned  systems. 


Figure  199.  The  Learning  Channel  films  an  early  version  of  ROBART  III  in  Building  F-36,  Seaside, 
circa  June  2001. 


144 


Mobile  Detection  Assessment  Response  System-Exterior  (1992-201 1 ) 

The  MDARS-Exterior  program  was  initiated  to  extend  the  robotic-security  and  automated 
inventory-control  concepts  of  the  MDARS-Interior  program  into  the  realm  of  semi-structured  outdoor 
environments  (i.e.,  improved  roads  and  defined  fence  lines).  Typical  application  sites  included 
storage  yards,  arsenals,  tank  farms,  airfields,  rail  yards,  port  facilities,  and  airfields.  Inventory  control 
consisted  of  verifying  the  contents  of  closed  structures  (i.e.,  warehouses,  bunkers,  igloos)  without 
need  for  opening,  as  well  as  discrete  items  routinely  stored  outdoors  (aircraft,  vehicles,  equipment, 
etc.).  A  centralized  database  of  high-value  inventory  was  compared  with  observed  inventory  as 
monitored  by  interactive  RFID-tag  readers  on  board  the  patrolling  robotic  vehicles. 

In  1993,  the  Army  Program  Office  awarded  Robotic  Systems  Technology  (RST),  now  General 
Dynamics  Robotic  Systems,  a  Broad  Agency  Announcement  (BAA)  contract  to  develop  the  semi- 
autonomous  robotic  platforms  capable  of  outdoor  navigation.  These  Performance  Definition  and  Risk 
Reduction  (PDRR)  mobility  platforms  were  equipped  with  on-board  payloads  for  intruder 
detection/assessment,  barrier  assessment,  and  inventory  management  (Myers,  1994).  The  Multiple- 
Resource  Host  Architecture  (MRHA)  developed  by  NOSC  for  command  and  control  of  MDARS- 
Interior  was  upgraded  to  accommodate  exterior  vehicles  and  provided  as  government- furnished 
equipment  (GFE)  as  shown  in  Figure  200. 


Figure  200.  Originally  developed  for  the  MDARS-Interior  robots  (screens  1  and  3  at  left),  the  MRHA 
was  upgraded  in  1994  to  also  control  MDARS-Exterior  robots  (screens  2  and  4  at  right)  (adapted 
from  Cory,  Everett,  &  Heath-Pastore,  1 998). 

The  design  of  the  MDARS-Exterior  system  was  driven  by  a  number  of  definitive  characteristics  of 
the  application  domain: 

•  The  MDARS  system  had  to  function  as  a  key  component  of  an  overarching  security  strategy 
that  included  fixed-detection  systems  and  human  security  guards. 
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•  The  patrol  coverage  of  multiple  mobile  robotic  platforms  had  to  minimize  opportunities  for 
undetected  intrusion,  even  by  insider  threats. 

•  The  operating  environment  required  navigational  capabilities  that  responded  to  dynamic 
events  (e.g.,  pedestrians  or  other  vehicles)  in  a  reasonably  structured  environment. 

Based  on  commercially  available  BobCat  components,  the  mobility  base  shown  in  Figure  201  was 
a  four-wheel  hydrostatic-drive,  diesel-powered  vehicle  (Myers,  1995).  This  BAA  prototype  weighed 
1700  pounds  and  measured  84  inches  long  by  35  inches  high  by  50  inches  wide,  with  an  8-inch 
ground  clearance  (Figure  201).  Designed  by  Ron  Griffin  of  Attraction  Services  under  subcontract  to 
RST,  the  mechanical  construction  of  this  custom  vehicle  proved  so  rugged  and  reliable  that  three 
such  BAA  prototypes  supported  continued  UXV  development  in  San  Diego  for  many  years.  Unlike 
the  cumbersome  GATORS/TOV retrofit  design,  all  mobility  functions  (forward,  reverse,  velocity, 
steering,  and  braking)  were  implemented  via  hydraulic  control  of  speed  and  direction. 


Figure  201 .  Designed  and  constructed  by  Ron  Griffin,  the  MDARS  BAA  mobility  prototype 
undergoes  preliminary  assessment  at  RST  facilities  alongside  the  Westminster  Municipal  Airport 
(background)  in  Maryland. 

For  improved  dead-reckoning,  however,  an  Ackerman-steered  design  was  chosen  over  the  skid- 
steer  BobCat  arrangement.  With  a  low  center  of  gravity  for  maximum  stability,  the  MDARS-Exterior 
vehicle  was  required  to  operate  over  paved,  gravel,  and  unimproved  roads  at  speeds  up  to  9  miles  per 
hour,  while  automatically  avoiding  obstacles  and  breaches.  The  four-wheel  hydrostatic-drive 
configuration  was  powered  by  a  24-horsepower  three-cylinder  diesel  engine  with  a  24-volt  alternator 
and  integral  power  steering  pump.  GDRS  demonstrated  autonomous  navigation  of  this  platform  in 
1996,  with  collision  avoidance  added  the  following  year. 
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Figure  202.  RST  vice  president  Mark  Del  Giorno  evaluates  early  autonomous  GPS-based  route 
execution  at  the  company’s  Westminster  facilities  prior  to  the  addition  of  collision  avoidance,  circa 
1996. 


The  collision-avoidance  strategy  incorporated  a  two-tier  layered  approach.  A  long-range  (0-100 
feet)  low-resolution  scanning  laser  provided  broad  first-alert  obstacle-detection  coverage,  while 
shorter-range  (0-30  feet  typical)  higher  resolution  sensors  (ultrasonic  sensors  and  stereo  vision)  were 
used  for  more  precise  obstacle  avoidance.  The  intruder-detection  system  (IDS)  employed  millimeter- 
wave  radar  (Figure  203)  and  forward-looking  infrared  (FLIR)  sensors  to  detect  the  movement  of 
intruders  within  a  6.6-  to  328-foot  range,  with  360-degree  coverage  around  the  platform  (Cory, 
Everett,  &  Heath-Pastore,  1998).  The  product-assessment  and  barrier-assessment  systems  used  RFID 
technology  to  automatically  inventory  tagged  items  and  interrogate  instrumented  locking  devices  on 
weapon-storage  igloos. 


Figure  203.  a)  Front  view  of  the  turret-mounted  IDS  radar,  b)  Rear  view  of  the  IDS  radar  with  cover 
removed. 
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The  MDARS-Exterior  BAA  system  underwent  Technical  Feasibility  Testing  at  Aberdeen  Proving 
Ground,  MD,  during  the  fall  of  1998  (Figure  204).  Repetitive  test  scenarios  were  run  to  fully  evaluate 
all  aspects  of  vehicle  command  and  control,  real-time  localization,  collision  avoidance,  and  intruder 
detection  in  both  daytime  and  nighttime  settings.  Acquired  by  General  Dynamics,  RST  became 
General  Dynamics  Robotic  Systems  (GDRS)  in  August  1999.  In  May  2000,  a  second  round  of 
Technical  Feasibility  Testing  by  the  U.S.  Army  Aberdeen  Test  Center  (ATC)  began  at  Edgewood, 
MD.  Based  upon  the  results  and  user  input  from  the  Military  Police  School  at  Fort  Leonard  Wood, 
MO,  PM  PSE  awarded  a  follow-on  contract  to  GDRS  in  January  2002  for  a  System  Development 
and  Demonstration  (SDD)  prototype. 


Figure  204.  The  MDARS-Exterior  development  team  at  Aberdeen  Proving  Ground,  circa  1998  (left  to 
right:  GDRS  president  Scott  Myers,  Dr.  Peter  Burt,  Mark  Del  Giorno,  Jay  Kurtz,  Phil  Cory,  Brian 
Frederick,  MDARS-Exterior  Program  Manager  Tracy  Pastore,  Robin  Laird,  MDARS  Technical 
Director  Bart  Everett,  Kevin  Bonner,  Bert  Farabaugh,  Gooitzen  van  der  Wal,  and  Robbie 
Mandelbaum). 

During  this  time,  SSC  Pacific  conducted  parallel  exploratory  development  efforts  to  expand  the 
force-protection  capabilities  of  MDARS.  These  included  the  addition  of  a  non-lethal  gun  pod  and 
marsupial  carriers  that  could  deploy  smaller  tracked  robots  and  vertical-takeoff-and-landing 
unmanned  aerial  vehicles  for  close-up  investigation  of  off-path  incidents.  Having  served  their 
original  purpose,  the  three  MDARS-Exterior  BAA  platforms  were  turned  over  to  the  Unmanned 
Systems  Group  in  San  Diego  to  support  application-payload  development  for  a  number  of  follow-on 
UGV  projects  (Figure  205). 
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Figure  205.  MDARS-Exterior  project  manager  Daniel  Carroll  explains  the  BAA  prototype  (lower  left) 
to  the  History  Channel  film  crew  in  front  of  Building  F-36,  Seaside  in  June  2003.  A  second  MDARS 
BAA  platform  equipped  with  the  iStar  UAV  launch/recovery/refueling  payload  is  shown  just  inside  the 
building  at  upper  right. 


Developmental  testing  under  the  System  Development  and  Demonstration  (SDD)  phase  of  the 
MDARS-Exterior  system  took  place  at  the  Hawthorne  Army  Depot  (HWAD),  the  world’s  largest 
army  depot  (147,000  acres),  located  in  west  central  Nevada  (Figure  206).  Four  MDARS  vehicles  were 
operated  by  base  security  forces  for  Early  User  Appraisal  (EUA)  of  the  installed  system.  These 
“Patrol  Unit  Vehicles”  provided  security  for  270  storage  igloos  grouped  in  15  bunker  sites,  as  further 
described  by  Carroll,  Nguyen,  Everett,  &  Frederick  (2005).  Upon  the  successful  completion  of  EUA, 
an  MDARS-Exterior  production  contract  was  awarded  to  GDRS  in  2007. 


Figure  206.  Jack  Gonzales  refuels  the  MDARS-Exterior  robot  during  Early  User  Appraisal  at 
Hawthorne,  the  world’s  largest  army  depot  (147,000  acres),  circa  2005. 
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Several  factors,  particularly  system  complexity,  contributed  to  significant  delays  during  the  first 
several  months  of  production.  To  help  overcome  these  challenges,  Rachel  TenWolde  was  assigned  to 
provide  onsite  production  oversight  at  GDRS  in  2008.  During  the  next  12  months,  she  helped 
implement  manufacturing  processes  to  improve  inspection,  assembly,  quality  assurance, 
configuration  management,  test,  and  evaluation.  She  also  helped  resolve  several  tense  disagreements 
between  GDRS,  the  Army,  and  the  Defense  Contract  Management  Agency,  fostering  a  more 
cooperative  working  environment  for  all.  GDRS  completed  five  MDARS-Exterior  SDD  systems  that 
were  delivered  to  HWAD  for  safety,  performance,  and  endurance  testing  in  2009  (Figure  207). 


Figure  207.  a)  One  of  five  MDARS-Exterior  production  vehicles  undergoes  final  checkout  at  GDRS 
prior  to  delivery  to  HWAD,  circa  2009.  The  upgraded  MRHA  operator  console  at  HWAD,  circa  2009. 

In  2010,  one  of  the  MDARS  SDD  vehicles  was  transferred  from  HWAD  to  the  Department  of 
Energy’s  Nevada  National  Security  Site  (NNSS)  in  Nye  County,  NV  (Figure  208a).  In  201 1,  the 
Army  Office  of  the  Provost  Marshal  General  (OPMG)  determined  MDARS-E  was  unaffordable,  and 
the  Center  for  Army  Analysis  (CAA)  predicted  a  relatively  small  long-term  return  on  investment. 
The  remaining  four  systems  under  evaluation  at  HWAD  were  consequently  transferred  to  NNSS  as 
well  (Figure  208b).  By  the  end  of  that  year,  all  five  MDARS-E  vehicles  were  providing  around-the- 
clock  site  patrols  (Davis,  2011). 


Figure  208.  a)  The  first  of  five  MDARS-Exterior  vehicles  is  shown  on  patrol  at  the  Nevada  National 
Security  Site,  circa  201 1 .  b)  A  film  crew  documents  the  arrival  of  the  four  additional  MDARS-Exterior 
systems  at  NNSS. 
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Estimated  savings  at  NNSS  included  $1  million  in  annual  guard- force  labor  and  equipment- 
maintenance  costs,  plus  approximately  $6  million  in  infrastructure  costs  associated  with  remote  areas 
(lights,  towers,  cameras,  motion  sensors,  and  cable  trenching/installation).  Lack  of  spares  and 
contractor  support,  however,  eventually  reduced  the  number  of  functional  MDARS  units  to  zero,  and 
all  remaining  assets  were  subsequently  placed  in  storage  at  Sandia  National  Laboratories  in 
Albuquerque,  NM. 

In  2012,  PM-FPS  identified  a  UGV  need  in  JUONS  0315  as  a  component  of  integrated  Entry 
Control  Point,  and  initiated  modernization  under  MDARS  Increment  II  for  affordability, 
sustainability,  and  adaptability  (Figure  209).  See  later  Combat  Outpost  Surveillance  and  MDARS-II 
sections  under  Project  Summaries,  Volume  3. 


1  Broad  Agency  Announcement 
award  to  investigate  technologies 
and  build  brass  board  prototype 


2  Conducted  Technical 
Feasibility  Test  at  Aberdeen  PG. 
MO 


3  Conducted 
successful  MS  Bto 


Figure  209.  Production,  development,  and  testing  timeline  of  MDARS-Exterior,  Generations  1  and  2. 

Remote  Vehicle  Attitude  Awareness  (1 990-1 992) 

One  of  the  critical  technology  components  of  teleoperated  UGV  development  is  the  human- 
machine  interface,  for  which  a  key  issue  is  the  sufficient  operator  understanding  of  the  remote- 
vehicle  attitude  (Pastore,  1994).  Early  roll-over  accidents  attributed  to  insufficient  attitude  cues  had 
been  observed  and  documented  by  both  Sandia  National  Laboratories  (McGovern,  1990)  and  NOSC 
(Aviles  et  al.,  1990).  While  a  UGV  operator  primarily  receives  remote  vehicle-attitude  cues  from 
video  feedback,  some  control  stations  are  further  equipped  with  a  vehicle  pitch-and-roll  indicator. 
Historically,  however,  this  type  of  an  assistive  feedback  did  not  seem  to  help  operators  understand 
and  react  to  rapidly  changing  vehicle  attitude  while  traversing  rough  terrain  (Aviles  et  al.,  1990). 

The  Remote  Vehicle  Attitude  Awareness  project  managed  and  executed  by  Tracy  Heath  Pastore 
sought  to  develop  an  improved  human-machine  interface  that  provided  sufficient  vehicle-attitude 
feedback  for  improved  teleoperation,  with  minimal  impact  upon  operator  workload.  A  literature 
review  was  conducted  of  the  human  orientation  system,  specifically  sensor  stimulation  and  anatomic 
sensor  data  processing,  with  particular  focus  on  reports  compiled  by  the  aviation  medical  community 
(Gillingham  &  Wolfe,  1986).  Potential  methods  for  presenting  vehicle  attitude  to  the  operator  to 
stimulate  the  human  orientation  system  were  identified  and  explored.  The  selected  method  involved 
referencing  the  remote  visual  sensors  to  the  Earth’s  gravitational  field  (Pastore,  1994). 
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When  locally  operating  a  manned  vehicle,  the  driver  collects  and  processes  a  variety  of  sensor 
inputs  that  collectively  provide  a  reasonably  accurate  understanding  of  real-time  vehicle  orientation. 
When  remotely  driving  a  UGV,  however,  appropriate  orientation  cues  must  be  recreated  (or  artificial 
cues  generated)  at  the  control  console  to  achieve  the  same  effect.  For  this  effort,  the  human 
subsystems  were  grouped  into  the  following  categories:  1)  visual,  2)  proprioceptive  (vestibular  and 
non-vestibular),  3)  mechanoreceptive,  and  4)  auditory.  Methods  of  presenting  orientation  cues  were 
identified,  then  classified  as  a  stimulant  of  one  of  the  three  categories  of  human  orientation 
subsystems  as  shown  in  Table  3  (Pastore,  1994). 


Table  3.  Potential  methods  of  vehicle-attitude  feedback  (adapted  from  Pastore,  1994). 


Visual  Cues 

Proprioceptive  Cues 

Auditory  Cues 

Stereo  displays 

Motion  seat 

Binaural  hearing 

Color  displays 

Force  reflective 

Warning  sounds 

Wide  FOV  displays 

Alarm  sounds 

Gravity  referenced  displays 

Vehicle  attitude  indicator 

The  actuated  two-degree-of- freedom  platform  shown  in  Figure  210  was  designed  to  gravity 
reference  the  remote  color,  stereo-vision,  and  binaural-audio  sensors  employed  on  a  HMMWV 
teleoperator  testbed  used  by  the  NOSC  HI  Lab.  The  existing  HMMWV  sensors  were  relocated  to  the 
prototype  referencing  platform,  which  was  installed  between  the  front  driver  and  passenger  seats, 
with  the  sensors  at  an  elevation  equal  to  the  average  eyes/ears  height  of  a  sitting  6-foot-tall  human 
(Diffrient  et  al.,  1974).  The  hood  of  the  remote  vehicle  appeared  in  the  video  image  and  served  as  an 
artificial  horizon  indicator,  providing  the  operator  with  vehicle  pitch  and  roll  information. 


Figure  210.  The  electric  cam  actuators  located  under  the  rotating  plate  provided  a  spring-opposed 
linear  pushing  action  to  achieve  the  required  angular  range  of  motion.  The  referencing  platform  did 
not  stabilize  the  remote  sensors;47  it  responded  only  to  changes  in  terrain  slope,  and  not  to  transient 
bumps  (adapted  from  Pastore,  1994). 


47 


A  stabilization  scheme  would  forfeit  information  on  road  roughness. 
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Mechanical  springs  were  positioned  on  each  axis  of  the  referencing  platform  to  reduce  the  lateral 
motion  introduced  into  the  system  by  the  push-rod  and  cam  actuators.  Each  axis  was  powered  by  a 
DC  motor  coupled  to  a  harmonic  gear  head,  with  limit  switches  to  prevent  rotational  displacement  in 
excess  of  the  platform's  physical  limits.  The  system  also  integrated  velocity  data  from  a  solid-state 
angular  rate  sensor,  and  had  a  pendulum  sensor  that  was  used  to  recalibrate  the  integrated  rate  sensor 
data  every  60  seconds.  A  two-DOF  inclinometer  provided  platform  pitch-and-roll  measurement 
(Figure  211). 


Figure  21 1 .  Tracy  Heath  Pastore  (left)  and  Dr.  Jerry  Fuqua  (right)  work  on  installing  the  gravity- 
referenced  platform  between  the  HMMWV seats  (lower  left)  at  the  NOSC  HI  Lab.  Note  black  2-DOF 
inclinometer  atop  the  platform. 

A  full-factorial  experiment  was  conducted  to  compare  the  effectiveness  of  a  UGV  equipped  with 
gravity-referenced  sensors  to  the  same  UGV  with  vehicle-referenced  sensors  (i.e.,  sensors  fixed  to 
the  vehicle  and  subject  to  the  same  motion).  The  control  electronics  were  designed  to  accept  voltage 
inputs  from  two  angular-position  sensors  or  from  two  hand-turned  potentiometers;  each  axis  had  a 
select  switch  for  the  input  source  (Pastore,  1994).  This  configuration  provided  for  efficient  switching 
between  gravity-  and  vehicle-referencing  methods,  and  facilitated  rapid  field  calibration.  The  test 
subject's  error  in  estimating  pitch  and  roll  angles  was  the  parameter  used  to  compare  the  effectiveness 
of  the  two  methods. 

The  test  scenario  was  designed  to  emulate  a  typical  remote-vehicle  driving  scenario  in  an 
unstructured  environment,  which  is  a  dynamic  situation  requiring  on-the-fly  operator  response, 
particularly  if  the  vehicle  is  traversing  rough  unfamiliar  terrain.  The  operator  must  be  fully  aware  of 
vehicle  attitude  at  all  times  to  determine  if  continued  operation  is  safe.  For  cost  and  safety  reasons, 
the  experiment  was  conducted  in  the  laboratory  using  field-collected  data.  The  HMMWV  was  taken 
to  Bellows  Air  Force  Station  in  Waimanalo,  HI,  and  driven  over  rough  dirt  roads  and  HMMWV- 
blazed  terrain  at  a  speed  of  7  ±  3  mph,  alternating  between  gravity-referencing  the  sensors  and 
vehicle-referencing  the  sensors. 

Figure  212  shows  the  data-collection  instrumentation.  Stereo  video  was  recorded  on  Recorder  1 
via  a  field-sequential  mixer,  monocular  video  was  recorded  directly  on  Recorder  2,  while  binaural 
audio  was  recorded  on  the  audio  tracks  of  Recorder  1.  The  voltage  signals  representing  vehicle  pitch 
and  roll  were  converted  to  corresponding  frequencies,  then  recorded  on  the  two  audio  tracks  of 
Recorder  2.  A  clapboard  was  used  at  the  beginning  of  each  clip  so  that  the  video  signals  on  Recorder 
1  and  Recorder  2  could  be  synchronized  during  editing.  The  data  were  edited  into  clips  and 
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transferred  to  two  optical  disks,  one  for  gravity-referenced  and  the  other  for  vehicle-referenced  data. 
The  optical-disk  medium  was  chosen  to  enable  the  random  selection  and  playback  of  the  clips  during 
testing. 


Figure  212.  Field  data-collection  instrumentation  (adapted  from  Pastore,  1994). 

Each  test  subject  participated  in  two  practice  sessions  followed  by  two  20-minute  test  sessions. 
Each  of  these  test  sessions  (as  well  as  the  practice  sessions)  featured  a  different  method  (gravity-  or 
vehicle-referencing)  of  presenting  UGV  attitude  information  via  video  displays  and  audio 
headphones.  Each  session  consisted  of  14  driving  scenarios,  45  seconds  in  length.  At  three  random 
points  during  each  45-second  scenario,  the  video  and  audio  stopped  and  the  test  subject  made  an 
estimate  of  the  current  HMMWV  pitch  and  roll  angles.  The  test  subject  expressed  his  or  her  estimates 
by  positioning  a  gimballed  model  vehicle  (see  Figure  4-2)  in  the  same  spatial  orientation  as  the 
observed  HMMWV. 


Figure  213.  a)  Project  engineer  Tracy  Heath  Pastore  is  wearing  stereo  flicker  glasses  and 
headphones  while  seated  at  the  test  operator  station,  with  the  video  monitor  at  far  left,  b)  Close  up  of 
the  gimballed  model  vehicle  used  to  input  operator’s  estimates  of  pitch  and  roll  angles,  based  on  the 
video  playback  (images  adapted  from  Pastore,  1994). 

The  test  subject  was  seated  at  the  appropriate  distance  behind  a  21 -inch  color  monitor  to  match  the 
39-degree  horizontal  field-of-view  of  the  video  cameras,  wearing  flicker  glasses  for  field-sequential 
stereo  viewing,  and  stereo  headphones  for  binaural  hearing  (Figure  213a).  The  gravity-referenced 
driving  clips  were  stored  on  one  optical  disk  and  the  vehicle-referenced  clips  were  stored  on  another. 
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The  clips  were  called  up  and  presented  to  the  test  subject  in  a  random  but  continuous  fashion.  The 
subject  positioned  the  model- vehicle  input  device  (Figure  213b),  then  depressed  a  hand-held  switch 
to  record  its  pitch  and  roll.  Actual  vehicle  pitch  and  roll  data  were  extracted  from  the  audio  tracks, 
converted  from  frequency  to  corresponding  voltage  values,  which  were  then  stored  in  an  input  data 
file.  Test  instrumentation  is  diagramed  in  Figure  214. 


Figure  214.  The  prerecorded  field  video  was  presented  to  the  test  subject’s  stereo  display  by  the 
optical  disk  player,  and  the  corresponding  perceived  vehicle  pose  was  captured  by  the  model- 
vehicle  operator-input  device  and  recorded  by  the  test  control  computer  (adapted  from  Pastore, 
1994). 

Twelve  volunteer  test  subjects  were  recruited  from  NOSC  employees  and  the  University  of  Hawaii 
at  Manoa,  none  of  whom  were  familiar  with  the  effort  (Table  4).  Each  test  subject  generated  42 
angle-estimation  data  points  during  each  of  two  test  sessions,  for  a  total  sample  size  of  1008  data 
points.  As  stated  earlier,  the  test  subject's  error  in  estimating  the  pitch  and  roll  angles  was  the 
parameter  used  to  compare  the  two  video-referencing  techniques.  Methods  of  statistical  inference 
were  employed  to  show  that  the  variance  in  the  angle-estimation  errors  was  attributable  to  the 
referencing  method  and  not  just  random  (chance)  variation  (Huntsberger,  1967;  Miller  &  Freund, 
1977).  It  was  clear  that  the  gravity-referencing  method  was  facilitating  more  accurate  roll  estimations 
than  the  vehicle-referencing  method. 
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Table  4.  Test  operator  characteristics  summary  (adapted  from  Pastore,  1994). 


Test 

Subject 

ID 

Current 

Occupation 

Number 

Years 

Driving 

Motion 

Sickness 

Susceptibility 

Previous 

HMMWV 

Experience 

Off-road 

Driving 

Experience 

1 

Student 

8 

Moderate 

Observer 

None 

2 

Admin  Assistant 

8 

Moderate 

Observer 

None 

3 

Engineer 

18 

Moderate 

Driver 

Minimal 

4 

Engineer 

30 

Moderate 

Driver 

Minimal 

5 

Engineer 

14 

Moderate 

Driver 

Minimal 

6 

Engineer 

29 

Minimal 

Observer 

Substantial 

7 

Engineer 

12 

Minimal 

Observer 

Minimal 

8 

Admin  Assistant 

21 

High 

Observer 

Minimal 

9 

Financial  Specialist 

16 

Minimal 

Observer 

None 

10 

Computer  Scientist 

14 

Minimal 

Observer 

Moderate 

11 

Mechanical  Technician 

40 

None 

Driver 

None 

12 

Student 

19 

Minimal 

Observer 

None 

Although  operator  awareness  of  vehicle  attitude  is  always  important,  it  becomes  critical  when  the 
vehicle  is  traversing  side  slopes  of  significant  angle,  and  thus  more  susceptible  to  rollover.  Figure 
215  displays  cell  mean  (average  estimation  error  for  each  actual  angle)  estimation  errors  plotted 
against  the  actual  roll  angle.  The  estimation  errors  associated  with  gravity  referencing  fall  within  a 
narrow  band  across  the  entire  side  slope  range,  are  consistent,  and  do  not  correspond  to  actual  side 
slope  angles.  On  the  other  hand,  the  estimation  errors  involved  with  vehicle  referencing  increase 
dramatically  as  the  actual  side-slope  angle  increases,  becoming  nearly  as  large  as  the  actual  side- 
slope  angles. 


ACTUAL  ANGLE  (deg) 

Figure  215.  Plot  of  roll-angle  cell  mean  estimation  errors  versus  actual  roll  angles  (adapted  from 
Pastore,  1994). 

In  summary,  statistically  significant  improvement  (with  a  99.9  percent  confidence  level)  was 
observed  in  test  operator  understanding  of  remote-vehicle  attitude,  both  pitch  and  roll,  when  the 
sensor  package  was  gravity-referenced  as  compared  to  when  it  was  vehicle-referenced.  The 
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improvement  in  operator  understanding  of  the  vehicle  roll  angle  with  gravity-referenced  sensors  was 
greater  than  that  observed  for  the  pitch  angle.  Experimental  results  also  showed  a  significantly  higher 
level  of  operator  confidence  in  vehicle  attitude  awareness  with  the  gravity-referencing  method.  A 
minimal  increase  in  operator  workload,  however,  was  also  documented  for  the  gravity-referencing 
method. 

Mission  Payload  Prototype  (1 992-1 994) 

See  later  Multipurpose  Security  and  Surveillance  Mission  Platform  (MSSMP)  project  summary. 

Air-Mob  He  Ground  Security  and  Surveillance  System  (1 992-1 996) 

In  the  early  1990s,  a  perceived  situational-awareness  disconnect  existed  between  the  capabilities  of 
UAV-bome  and  UGV-bome  remote-sensor  systems  employed  in  reconnaissance,  surveillance  and 
target  acquisition  (RSTA)  missions  (Murphy  et  al.,  1997).  The  energy  required  by  a  UAV  to  stay 
aloft  limited  the  amount  of  time  it  could  loiter  on  station  (hours),  while  a  UGV  could  remain  on 
station  for  a  much  longer  period  (days),  but  was  limited  in  speed  and  movement  over  rough  terrain. 
An  air-mobile  surveillance  system  with  vertical-takeoff-and-landing  (VTOL)  characteristics 
conceivably  combined  the  advantages  of  both  options  to  achieve  rapid  deployment  of  remote  sensors 
to  nearly  inaccessible  ground  locations  for  extended  surveillance. 

To  validate  this  concept,  the  Air  Mobile  Ground  Security  and  Surveillance  System  (AMGSSS) 
project  was  initiated  in  1992,  the  objective  being  to  provide  a  rapidly  deployable  extended-range 
surveillance  capability  for  force  protection  and  tactical  security.  The  program  was  supported  by  the 
U.S.  Army  Product  Manager-Physical  Security  Equipment  (PM-PSE)  and  the  Office  of  the 
Undersecretary  of  Defense  (Acquisition,  Tactical  Systems/Land  Systems).  In  1996,  AMGSSS 
acquired  additional  support  from  the  U.S.  Army  Military  Police  School  and  the  U.S.  Army  Infantry 
Center  Dismounted  Battlespace  Battle  Laboratory,  and  was  renamed  the  Multipurpose  Security  and 
Surveillance  Mission  Platform  (MSSMP).  Details  of  both  efforts  are  discussed  in  the  next  section. 

Multipurpose  Security  and  Surveillance  Mission  Platform  (1 996-1 997) 

The  AMGSSS/MSSMP  objective  was  to  provide  a  rapidly  deployable  extended-range  perch-and- 
stare  surveillance  capability  for  a  variety  of  missions  (Murphy  &  Bott,  1995;  Nguyen  et  al.,  1996; 
Murphy  et  al.,  1996): 

•  Force  protection  and  tactical  security 

•  Counter-drug  and  Border  Patrol  operations 

•  Detection  and  assessment  of  defensive  works  (minefields,  tank  traps) 

•  Remote  assessment  of  contaminated  areas  (chemical,  biological,  nuclear) 

•  RF  communications  relays. 

•  Fire  control  for  long-range  weaponry 

Initial  focus  was  on  the  RSTA  mission  requirements  for  force-protection  and  front-line  ground 
forces,  allowing  field  commanders  to  quickly  extend  their  information  gathering  perimeter  out  to  10 
kilometers.  The  rapidly  deployable/recoverable  air-mobile  surveillance  platform  could  be 
strategically  emplaced  to  provide  ground-level  autonomous  surveillance,  detection,  and  assessment 
capabilities.  Such  timely  collection  of  mission-essential  information  on  enemy  activity  and  terrain 
could  greatly  facilitate  force-protection  planning. 

Payload  mobility  was  provided  by  the  6-foot-diameter  Sikorsky  Cypher,  a  small  ducted- fan  VTOL 
UAV  weighing  some  300  pounds  (Figure  216).  The  conceptual  MSSMP  system  consisted  of  three 
air-mobile  ground-sensor  units,  a  HMMWV -mounted  base  station,  and  a  trailer  for  ground  transport. 
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The  air-mobile  platforms  could  be  collectively  deployed  as  a  barrier  or  independently  assigned  to 
monitor  specific  assets,  critical  routes,  or  choke  points,  providing  long-term  surveillance  without 
putting  personnel  at  risk.  Their  rapid  mobility  and  insensitivity  to  intervening  terrain  allowed  quick 
relocation  as  needed  in  response  to  changing  battlefield  conditions. 


Figure  216.  The  gasoline-powered  Sikorsky  Cypher  ducted-fan  UAV,  6  feet  in  diameter  and 
weighing  some  300  pounds,  was  selected  as  the  best  commercially  available  VTOL  platform  for 
near-term  MSSMP  demonstrations. 

The  portable  Mission  Payload  Package  (MPP)  shown  in  Figure  217  was  developed  for  the 
MSSMP  project  by  Center  engineers  and  scientists  (Murphy  et  al.,  1997).  Mounted  on  a  Transitions 
Research  Corporation  pan-and-tilt  unit,  each  surveillance-sensor  package  included:  1)  a  visible-light 
video  camera;  2)  an  infrared  video  camera;  3)  a  laser  rangefinder;  and,  4)  an  optional  acoustic  sensor. 
Communication  with  the  remote  operator  control  unit  was  over  the  standard-issue  SINCGARS  radio 
shown  at  image  lower  right. 48  Multiple  remote-surveillance  systems  (air-mobile  or  standalone), 
linked  over  legacy  low-bandwidth  tactical  radio  links,  were  to  be  supervised  by  a  single  operator, 
eliminating  the  need  for  full  time  attention  to  platform  operation. 


48  SINCGARS  was  the  Army’s  projected  least  common  denominator  Combat  Net  Radio  (Murphy  et  al.,  1997). 
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Figure  217.  The  ground-based  MPP  sensor  package  included  both  visible-light  and  infrared  video 
cameras,  a  laser  rangefinder,  and  an  optional  acoustic  sensor  collectively  mounted  on  a 
commercially  available  pan-and-tilt  unit. 

The  payload-weight  and  power-supply  capacities  of  the  air  vehicle,  along  with  the  bandwidth 
limitations  of  the  SINCGARS  communications  link,  imposed  several  constraints  on  the  surveillance 
system  design: 

•  The  sensor  package  had  to  be  small,  lightweight,  and  low  power. 

•  The  majority  of  sensor-data  processing  had  to  be  performed  at  the  vehicle  end  to  reduce 
bandwidth  and  RF  power  required  to  transmit  the  information. 

•  The  system  architecture  had  to  be  flexible  enough  to  allow  future  integration  of  more 
advanced  sensors. 

For  prototyping  purposes,  a  portable  Mission  Payload  Prototype  consisting  of  the  sensors  and 
remote  processors  was  assembled,  with  a  Windows-based  graphical  program  for  the  laptop  computer 
OCU  (Nguyen  et  al.,  1996).  This  approach  allowed  system  development  to  proceed  independently  of 
the  Cypher’s  progress.  Demonstrations  of  the  remote  sensing  subsystem  were  conducted  with  the 
MPP  acting  as  a  surrogate  MSSMP  vehicle,  providing  the  security  and  surveillance  functionality 
without  the  mobility.  The  sensors  and  processors  were  later  duplicated  and  packaged  into  a  Cypher- 
mounted  pod  (Figure  218).  The  MPP  proved  so  valuable  during  field  tests  that  it  continued  to  play  a 
role  in  subsequent  system  demonstrations,  serving  as  a  second  air-mobile  unit,  a  UGV-mounted  unit, 
or  a  man-portable  sensor  package. 
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Figure  218.  The  sensors  and  processors  employed  on  the  MPP  shown  in  Figure  217  were  adapted 
for  incorporation  into  an  airborne  surveillance  pod  mounted  atop  the  Sikorski  Cypher  UAV. 

The  sensor  suite  included  a  Cohu  visible  light  video  camera  with  a  Canon  10:1  zoom  lens  and 
2X  range  extender,  an  Inframetrics  InfraCam  FLIR  with  a  100-millimeter  lens,  a  Reigl  Lasertape 
rangefinder,49  and  an  interface  for  an  optional  acoustic  sensor  (Murphy  et  ah,  1996;  1997).  Due 
to  the  limited  RF  bandwidth,  most  sensor  processing  was  performed  by  the  payload,  as  for 
example  acoustic  and  visual  motion  detection  used  to  detect,  identify,  and  locate  targets  of 
interest.  Preprogrammed  responses  upon  detection  included  an  alert  to  the  operator,  automatic 
transfer  of  a  static  image,  streaming  video,  and/or  perceived  range  to  target. 

The  prototype  operator  control  unit  was  a  laptop  computer  running  a  graphical  Windows  program 
(Figure  219).  Commands  to  the  remote  sensors  were  initiated  using  the  built-in  keyboard  and 
pointing  device,  while  data  and  images  sent  back  were  displayed  on  the  laptop's  color  monitor. 
Communication  between  all  remote  payload  subsystems  and  the  control/display  station 
employed  a  fully  connected  Ethernet  TCP/IP  radio  network  with  auto-relaying  capability.  Given 
the  low-bandwidth  nature  of  the  SINCGARS  radios,  the  real-time  transmission  of  high-resolution 
data  streams  typically  required  for  360-degree  remote  surveillance  was  not  technically  feasible. 


File  Set  Up  Mode  View  Help 


|7  Pan  Scan 
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Figure  219.  The  MSSMP  user  interface  developed  by  Hoa  Nguyen  created  a  mosaic  of  imagery  that 
allowed  the  operator  to  virtually  pan  and  tilt  in  intuitive  fashion.  Other  modes  allowed  certain  high- 
interest  sectors  only  to  be  continuously  updated  while  the  remainder  remained  static. 


49  Good  to  about  800  meters,  the  Reigl  Lasertape  was  a  low-cost  rangefinder  surrogate  for  demonstration  purposes. 
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Instead,  an  innovative  step-and-stare  technique  was  employed  to  reposition  the  camera  through 
consecutive  segments  in  both  pan  and  tilt,  during  each  of  which  a  single  frame  of  time-stamped  video 
was  captured,  digitized,  and  transmitted  back  to  the  OCU.  Referring  again  to  the  top  of  the  display  in 
Figure  219,  this  stream  of  images  was  recombined  into  a  mosaic  that  allowed  the  operator  to  virtually 
pan-and-tilt  in  real  time,  even  though  the  actual  images  being  viewed  may  have  been  several  seconds 
old.  This  minor  latency  was  operationally  tolerable  for  remote  surveillance,  however,  especially  since 
the  operator  could  select  a  continuous  feed  of  any  specific  view,  while  platform-executed  motion- 
detection  algorithms  monitored  the  remainder. 

In  May  1996,  the  MSSMP  system  was  successfully  demonstrated  in  a  simulated  counter-drug 
operation  at  the  Military  Police  School,  Fort  McClellan,  AL.  The  man-portable  sensor  package 
(mounted  on  a  HMMWV  vehicle-of-opportunity)  and  the  Cypher-mounted  sensor  package  (Figure 
220)  simultaneously  operated  over  the  same  radio  network  (Murphy  et  al.,  1997).  GPS  waypoints 
programmed  into  the  OCU  prior  to  the  beginning  of  a  mission  were  used  by  the  Sikorski  Cypher 
autopilot  to  navigate  from  take-off  to  landing.  In  approximately  1  hour,  Sikorsky  successfully  taught 
an  untrained  soldier  to  program  and  fly  an  actual  mission,  using  a  simulator  that  allowed  operators  to 
plan  and  load  waypoints,  then  exercise  in  simulation. 


Figure  220.  The  MSSMP  system  demonstrated  in  May  1996  at  Fort  McClellan,  AL,  consisted  of  a 
Cypher-mounted  sensor  suite  (foreground)  and  an  MPP  attached  to  the  top  of  a  HMMWV 
(background  left). 

A  total  of  1 8  flights  were  conducted,  during  which  various  mission  payloads  were  used  on  both  the 
air-mobile  unit  and  the  HMMWV -mounted  MPP,  to  include  both  visible  and  infrared  (FLIR)  video 
cameras,  a  laser  rangefinder,  a  smoke/gas  dispersion  system,  a  laser  rangefinder/  target  designator. 
Specialized  mechanisms  were  used  to  carry  and  emplace  small  equipment,  such  as  a  communications 
relay  or  tactical  security  sensors.  It  took  an  average  of  just  10  minutes  to  change  the  vehicle  payload 
using  a  standard  socket  set  and  wrenches.  Due  to  the  payload-weight  limitation  of  the  Cypher 
vehicle,  however,  the  FLIR  and  laser  rangefinder  could  not  be  carried  on  board,  but  were  employed 
on  the  MPP  (see  again  Figure  220). 

In  January  1997,  the  MSSMP  system  was  showcased  in  a  Military  Operations  in  Urban  Terrain 
(MOUT)  scenario  at  the  Dismounted  Battlespace  Battle  Laboratory,  Fort  Benning,  GA.  Tactical 
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reconnaissance  support  was  demonstrated  with  the  Sikorski  Cypher  flying  down  streets  and  looking 
into  windows  to  provide  forward  reconnaissance  for  advancing  troops  (Figure  221a),  then  performing 
fixed-site  surveillance  after  landing  on  the  roof  of  a  three-story  building.  The  air  vehicle  also  dropped 
a  simulated  radio  relay  on  top  of  a  two-story  building  (Figure  221b)  and  a  miniature  intrusion- 
detection  sensor  in  an  open  field  (Murphy  et  al.,  1997). 


Figure  221 .  a)  The  air-mobile  MSSMP  vehicle  flew  just  ahead  of  advancing  troops,  providing  forward 
reconnaissance  during  the  January  1997  demonstrations  at  the  Fort  Benning  MOUT  site,  b)  The 
Sikorski  Cypher  dropped  a  simulated  communications  relay  on  the  roof  of  a  two-story  building. 

Sensor  Motor  Transformation  (1993-1997) 

The  ability  to  detect  and  respond  to  targets  or  obstacles,  both  moving  and  stationary,  is  critical  for 
unmanned  vehicle  applications.  While  this  capability  is  readily  achieved  by  most  animate  systems,  in 
the  early  1990s  it  was  still  fairly  difficult  to  implement  on  their  artificial  analogs.  The  Sensor-Motor 
Transformation  project  attempted  to  show  that  efficient  extensible  solutions  to  the  target  acquisition 
and  discrimination  problem  could  be  achieved  by  emulating  the  mechanisms  employed  by  biological 
systems.  In  nature,  visual  motion  provides  the  basis  for  visual  target  detection,  acquisition,  tracking, 
and  pursuit  (Blackburn  &  Nguyen,  1994a). 
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Figure  222.  Visual-motor  functions  and  relationships  as  applied  to  the  ModBot  track-and-follow 
behavior  illustrated  in  Figure  223a  (adapted  from  Blackburn  &  Nguyen,  1995). 

For  this  reason,  our  pursued  approach  emulated  natural  strategies  (Figure  222),  identifying  the  3D 
spatial  location  of  a  unique  source  of  motion  from  the  optic  flow  created  by  an  independently  moving 
object  in  the  visual  field  (Blackburn  &  Nguyen,  1995).  The  previously  discussed  ModBot  system, 
which  had  a  monocular  video  camera  on  a  pan-and-tilt  mechanism  (Figure  223),  served  as  the 
software-development  platform  for  the  biologically  based  visual-motor  control  algorithms.  The 
primary  target  information  of  interest  was  detected  video  motion,  and  because  such  could  arise  from 
either  target  or  platform  movement,  an  effective  method  of  motion  segmentation  was  required. 


Figure  223.  A  laser  pointer  was  added  to  the  camera  pan-and-tilt  unit  of  the  ModBot  by  Hoa  Nguyen. 
This  configuration  was  eventually  replaced  by  the  stereo  Zebra  Vergence  system  offered  by 
Transitions  Research  Corporation  (TRC),  Danbury,  CN,  shown  later  in  Figure  225. 
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The  motion-analysis  algorithms,  developed  under  earlier  work  by  Blackburn  et  al.  (1987),  were 
enhanced  for  this  effort  to  allow  separation  of  unique  target  motion  from  the  collateral  optic  flow  due 
to  robot  movement  through  a  visually  complex  environment.  By  way  of  example,  center-surround 
receptive  fields  were  used  to  minimize  optic  flow  created  by  the  moving  robot  and  enhance  that  of 
target  motion.  Testing  was  performed  in  a  large  room  with  an  open  work  area  of  32  by  18  feet 
(Figure  223).  From  a  resting  start  position,  the  ModBot  turned  and  moved  forward  in  pursuit  of  a 
human  walking  into  its  visual  space,  with  obstacle  avoidance  disabled  to  allow  the  robot  to  approach 
the  cluttered  desk.50 


Figure  224.  Specifically  configured  for  the  Sensor-Motor  Transformation  project,  the  visually  guided 
ModBot  autonomously  trails  electrical  engineer  Theresa  Tran  as  she  moves  about  in  Room  115  of 
Building  622  (adapted  from  Blackburn  &  Nguyen,  1994a). 

Challenges  associated  with  this  application  included  the  following  (Blackburn  &  Nguyen,  1994a): 

1 .  The  complexity  of  the  background 

2.  The  proximity  of  the  target 

3.  The  velocity  of  the  target  image  on  the  visual  field 

4.  The  absence  of  unique  distinguishing  features  associated  with  the  target 

5.  The  limitations  of  on-board  processing  power  and  energy  resources 

The  underlying  strategy  was  explained  by  Blackburn  &  Nguyen  (1995): 

“Natural  mechanisms  have  been  successful  at  all  levels  of  complexity,  and 
they  achieved  additional  complexity  and  capability  by  maintaining  and 
modulating  more  elementary  functions.  Since  nature  began  with  the 
production  of  very  simple  organisms  living  successfully  within  protected 
ecologies,  so  might  we  begin  in  our  design  of  artificial  intelligence  machines 
with  very  simple  functions  appropriate  for  a  particular  environment.” 


50  With  obstacle  avoidance  enabled,  the  robot  tended  to  approach  the  target  more  cautiously,  until  a  sufficiently  clear 
approach  path  was  perceived  (Blackburn  &  Nguyen,  1994a). 
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In  1994,  the  machine-vision  system  shown  in  Figure  223  was  upgraded  to  a  stereo  pan,  tilt  and 
vergence  mechanism  developed  by  TRC  (Figure  225),  which  was  used  to  control  both  the  ModBot 
and  a  five  degree-of-freedom  stationary  manipulator  arm.  These  two  efforts  combined  vision, 
navigation,  and  manipulation  to  study  learning  of  cross-modal  sensor-sensor  and  sensor-motor 
sequences  (Blackburn  &  Nguyen,  1994).  Performance  of  the  vergence  mechanism  was  rather 
disappointing,  however,  and  consequently  did  not  add  much  to  the  perception  functionality  (Nguyen, 
2016). 


Figure  225.  The  TRC  Zebra  Vergence  stereo  pan,  tilt,  and  vergence  unit  was  temporarily  mounted 
on  the  top  ring  of  the  ModBot  to  support  the  Sensor-Motor  Transformation  project. 

The  Sensor-Motor  Transformation  approach  emulated  natural  strategies,  identifying  the  3D  spatial 
location  of  a  unique  source  of  motion  from  the  optic  flow  created  by  an  independently  moving  object 
in  the  visual  field  (Blackburn  &  Nguyen,  1995).  The  algorithms  of  active  perception  employed  both 
reflexive  and  adaptive  mechanisms  (Blackburn  &  Nguyen,  1994).  Reflexive  mechanisms  provided 
low-level  fault-tolerant  solutions  for  target  detection,  segmentation,  and  obstacle  avoidance,  while 
adaptive  mechanisms  provided  intrinsically  modifiable  solutions  to  difficult  problems  such  as  eye- 
hand  calibration  and  target  discrimination. 

Man-Portable  Networked  Sensor  Package  (1 997-1 999) 

Spawned  by  the  MSSMP  project  in  the  latter  half  of  1997,  the  Man-Portable  Networked  Sensor 
Package  (MPNSS)  effort  focused  on  development  of  ruggedized  lightweight  sensor  system  for 
unattended  battlefield  applications.  Experience  gained  at  the  Ft.  Benning  MOUT-site  demonstrations 
earlier  that  year  suggested  that  by  using  the  MSSMP  wireless-network  architecture,  but  without  the 
cost  of  the  air-mobile  platforms,  a  network  of  inexpensive  man-portable  sensor  units  could  make 
significant  contributions  to  military  operations  in  urban  terrain.  An  artist’s  concept  drawing  is  shown 
in  Figure  226. 
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Figure  226.  Artist’s  concept  of  a  ruggedized  Man-Portable  Networked  Sensor  Package  for 
unattended  battlefield  applications  (adapted  from  Bryan,  Nguyen,  &  Gage,  1998). 

The  resulting  MPNSS  system  consisted  of  a  scalable  suite  of  smart  sensors  configured  into  man- 
portable  packages  that  could  be  distributed  throughout  a  surveillance  area,  linked  by  a  wireless- 
network  communications  architecture.  The  majority  of  data  processing  was  performed  at  the  remote 
sensor  node,  with  control/output  data  to  and  from  the  nodes  available  to  users  throughout  the 
network.  This  remote-processing  approach  reduced  the  bandwidth  and  power  consumption  required 
to  transmit  the  information,  as  decisions  made  by  the  remote  computers  ensured  only  useful  data  was 
transmitted.  The  flexible  system  architecture  enabled  scalability,  in  that  a  given  sensor  or  network  of 
sensors  could  easily  be  configured  to  satisfy  specific  mission  requirements. 

By  way  of  example,  sensor  packages  could  consist  of  a  helmet-mounted  video  camera,  an 
unattended  ground  sensor,  or  a  suite  of  integrated  sensors.  The  baseline  sensor  suite  shown  in  Figure 
227a  provided  the  following  functionality: 

•  Video/images  for  visual  target  classification 

•  Video  motion  detection 

•  Image  enhancement 

•  Acoustic  alerts  and  acoustic  classification 

•  Sensor/image  processing  for  bandwidth  reduction 

•  GPS,  heading,  and  range  data  processing  for  absolute  target  positioning 


Figure  227.  Tested  MPNSS  sensors  included:  a)  Integrated  MPNSS  sensor  package;  b)  Northrop 
Grumman’s  Unattended  Ground  Sensor,  c)  Tactical  Remote  Sensor  System  (TRSS)  ground  sensor. 
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Given  the  evolving  state  of  commercial  acoustic  detectors,  an  external  data  connection  was 
provided  for  an  optional  acoustic  sensor,  instead  of  physically  incorporating  one  into  the  MPNSS.  In 
late  1995,  a  prototype  air-deployed  acoustic  sensor  made  by  Northrop-Grumman,  a  small  ring  of 
three  microphones  with  custom  processing  hardware  and  a  serial  interface  (Figure  227b),  had  been 
connected  to  the  MSSMP  Mission  Payload  Prototype  and  tested  in  the  field.  Output  included  target 
azimuth,  target  type  (ground  vehicle,  jet,  helicopter,  etc.),  and  detection/classification  confidence. 
Subsequent  MPNSS  software  provided  programmable  filters  that  allowed  the  user  to  discard  certain 
types  of  sound  from  specified  azimuthal  areas. 

Legacy  sensor  systems  such  as  the  Tactical  Remote  Sensor  System  (TRSS)  shown  in  Figure  221c , 
the  Improved  Remotely  Monitored  Battlefield  Sensor  System  (IREMBASS),  and  the  Tactical 
Automated  Security  System  (TASS)  employed  a  standard  communications  protocol  ( SEIWG-05 )  for 
transmitting  sensor  data  over  VHF  point-to-point  radio  links.  A  small  VHF  transceiver  was 
embedded  in  the  baseline  MPNSS  sensor  package  to  deliver  this  data  to  the  payload  processor,  with 
associated  software  to  monitor  alerts  from  these  sensor  systems  and  reformat  the  data  into  IP  packets. 
This  data  was  multiplexed  and  distributed  across  the  network  to  all  users.51 

Three  low-power  PC/1 04  microcomputers,  interconnected  via  Ethernet  link,  processed  and 
reduced  the  raw  data  collected  before  passing  to  the  control/display  unit.  Each  computer  had  its  own 
Internet  address,  and  theoretically  could  be  thousands  of  miles  away,  connected  by  any  available  IP- 
based  infrastructure,  wired  or  wireless.  This  distributed  architecture  allowed  parallel  development  of 
the  various  subsystems  at  different  sites,  facilitated  debugging  by  substitution  of  any  processor  by  an 
equivalent  desktop  computer,  and  resulted  in  a  flexible  system  that  could  be  readily  expanded  or 
modified  (Nguyen  et  al.,  1996). 

The  payload  processor  communicated  with  the  control/display,  interpreting  and  executing  high- 
level  operator  inputs  to  generate  low-level  commands  to  the  various  sensor  subsystems.  It  also 
coordinated  the  flow  of  information  between  all  payload  computers,  monitoring,  filtering,  and 
consolidating  alerts  from  the  image  processor,  acoustic  detector,  and  legacy  sensor  gateway.  The 
Image  Processor  performed  video  motion  detection  and  image  enhancement  when  commanded, 
compressing  still  images  before  sending  them  to  the  control/display.  The  video  processor  was 
dedicated  to  real-time  video  compression  and  transmission,  using  a  Windows-95  CUSeeMe  software 
package 

The  MPNSS  communications  architecture  consisted  of  an  all-digital  adaptive  network  providing 
integrated  video,  voice,  and  data  services  as  shown  in  Figure  228  (Murphy  et  ah,  1997;  Martin  & 
Bryan,  1995).  The  physical  layer  for  IP-based  network  connectivity  was  provided  by 
Magnavox/SAIC  PCMCIA  Tactical  Communication  Interface  Modules  (TCIMs),  with  software 
modified  by  us,  over  SINCGARS  radios.  Beyond-line-of-sight  network  connectivity  was  established 
with  this  tactical  IP  data  link  during  operational  field  tests  in  1995.  As  effective  data  throughput  was 
low  (<  500  bps),  commercial  WLAN  technology  was  substituted  to  improve  system  performance 
until  military  communications  technology  had  further  advanced. 


51  A  more  detailed  description  of  MPNSS  sensors  is  provided  in  Murphy  et  ah,  1996. 
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UAV-mounled  Portable  sensor 

sensor  package  package 


Figure  228.  The  scalable  MPNSS  system  architecture  allowed  for  optimal  tailored  solutions  to 
varying  user  needs. 


The  MPNSS  user  interface  shown  in  Figure  229  was  a  Windows-based  software  program  with 
standard  windows,  menus,  buttons,  and  dialog  boxes  supporting  the  following  functionalities: 

•  Command  remote  sensors  to  perform  elementary  functions,  including  taking  target  snapshots 
(at  specified  zoom,  focus,  gain,  polarity,  azimuth  and  elevation,  etc.) 

•  Determine  target  position  and  measure  target  range 

•  Enhance  target  images  prior  to  transmission 

•  Initiate  live  compressed  video  of  target 

•  Program  complex  sequences  of  commands,  such  as  composing  image  panoramas,  acoustic 
filters,  and  motion  detection  at  various  critical  scene  points 

An  in-depth  discussion  of  how  these  commands  are  executed  can  be  found  in  Murphy  et  al.,  1996, 
with  details  of  the  Control/Display  features  presented  in  Nguyen  et  al.,  1996. 


b) 

Figure  229.  Sample  Control/Display  screens:  a)  Ft.  Benning  panorama  and  corresponding  aerial 
photo;  b)  Motion  detection  alert,  with  target  image  and  location  overlaid  on  map. 
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The  Control/Display  program  was  initially  developed  for  laptop  computers  (Figure  230a),  then 
subsequently  demonstrated  on  a  Xybemaut  soldier-wearable  computer  (Figure  230b),  and  the  Litton 
Handheld  Terminal  Unit  (Figure  230c).  The  wearable  computer  included  an  integrated  trackball,  a 
battery  pack  worn  around  the  waist,  and  a  small  head-mounted  display,  with  an  integrated  video 
camera,  microphone  and  earphone.  This  configuration  added  several  new  capabilities  to  the  system 
(Bryan,  Nguyen,  and  Gage,  1998): 

•  The  warfighter  could  access,  monitor,  and  control  remote  sensor  data  while  on  the  move. 

•  Voice  control  and  feedback  of  sensor  system  functions  provided  hands-free  operation. 

•  The  heads-up  display  presented  remote  sensor  data  within  the  warfighter’s  field  of  view. 

•  Video  from  the  head-mounted  camera  enabled  the  warfighter  to  function  as  a  mobile  sensor 
within  the  network. 


Figure  230.  Tested  MPNSS  control/display  configurations  included:  a)  a  laptop  computer, 
b)  a  wearable  computer,  and  c)  the  Dismounted  Soldier  System  Unit. 


In  January  1997,  the  prototype  MPNSS  baseline  sensor  package  was  demonstrated  in  the  same 
wireless  network  with  its  Cypher-mounted  counterpart  in  the  U.S.  Army  MOUT  facility  at  Fort 
Benning,  GA  (Murphy  et  al.,  1997).  The  system  provided  reconnaissance  support  for  advancing 
troops  and  security  surveillance  in  urban  terrain.  In  August  1997,  the  MPNSS  portable  baseline 
sensor  package  was  operated  in  stand-alone  configuration  during  MOUT  exercises  at  Camp 
Pendleton  Marine  Corps  Base,  CA.  Test  plans  were  developed  that  same  year  to  demonstrate  long¬ 
term  severe-weather  use  of  MPNSS  for  augmenting  base  security  measures  at  Naval  Air  Station 
Keflavik,  Iceland,  although  such  a  plan  was  never  implemented. 

DARPA  Tactical  Mobile  Robotics  (1997-2003) 

Initiated  by  Dr.  Eric  Krotkov  in  1997,  the  DARPA  Tactical  Mobile  Robotics  (TMR)  program  was 
managed  by  Lieutenant  Colonel  John  Blitch,  U.S.  Army  Reserve  (Krotkov  &  Blitch,  1999).  Dr. 

Doug  Gage  of  the  Adaptive  Systems  Branch  (D371)  served  as  TMR  chief  engineer  (Gage,  2000).  The 
overarching  goal  of  the  program  was  to  develop  versatile,  reactive,  and  robust  teams  of  small  robots 
that  could  operate  in  restricted  or  denied  areas  under  a  variety  of  realistic  conditions  (Anhalt  & 
Spofford,  1999).  These  envisioned  robotic  teams  were  intended  for  use  where  humans  could  not  go 
due  to  size  constraints,  dangerous  environments,  or  unstable  conditions. 

To  help  achieve  this  goal,  the  TMR  program  developed  a  number  of  small  man-portable  robotic 
prototypes  (Figure  231),  mission  payloads,  and  mobile  user  interfaces,  with  the  following  technology 
objectives  (Anhalt  &  Spofford,  1999): 

•  Robotic  team  configuration  and  control 

•  Robotic  collaboration 
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•  Robust  navigation,  localization,  and  mapping 

•  Sensor  fusion,  to  include  sensor  data  from  multiple  viewpoints 

•  Target  detection  and  tracking 


Figure  231 .  a)  The  Pioneer  AT  development  surrogate  from  ActivMedia,  Peterburough,  NH.  b)  The 
teleoperated  Urbie  robot,  developed  by  the  Real  World  Interface  (RWI)  division  of  IS  Robotics,  now 
iRobot  Corporation  (images  adapted  from  Arkin,  Collins,  &  Endo,  1999).  c)  A  teleoperated  Draper 
Laboratories  “ThrowBot”  prototype  in  expanded  mode. 

A  number  of  developmental  challenges  were  identified  (Gage,  2000): 

•  Acquiring  critical  component  technologies  to  support  the  desired  functionality 

•  Delivering  systems  small  enough  to  fit  in  a  rucksack  but  large  enough  to  perform  in 
challenging  environments,  with  sufficient  on-board  energy  for  meaningful  time  on  station 

•  Providing  supervised  autonomous  navigation  to  reduce  operator  burden  and  control 
bandwidth  requirements 

•  Performing  appropriate  systems  integration  and  implementation  to  achieve  a  useful  product 
Core  performers  of  Phase  I  (BAA  98-08  Part  A  -  Technology )  by  focus  area  were  (Gage,  2000): 

•  Mobility:  Massachusetts  Institute  of  Technology 

•  Sensors:  University  of  Michigan  (Figure  232) 

•  Perception:  Yale,  SRI  International 

•  Autonomy:  SRI  International,  Carnegie  Mellon  University,  Stanford  University,  University 
of  Southern  California,  and  Georgia  Tech 

•  Mission  Packages:  Foster-Miller 
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Figure  232.  The  University  of  Michigan’s  FLEXnav  PPE  precision  dead-reckoning  system 
implemented  on  a  Pioneer  AT. 

An  example  autonomous  behavior  implemented  by  Georgia  Tech  on  the  Pioneer  AT  surrogate  is 
shown  in  Figure  233,  where  the  robot’s  mission  was  to  enter  and  search  each  room  in  the  a  priori 
floorplan  of  Figure  234a,  looking  for  a  red-colored  target  object.  In  the  first  scenario,  the  robot 
started  at  far  left  in  the  expanded  image  at  lower  left,  travelled  down  the  hallway  past  Room  360, 
then  entered  the  adjacent  copy  room,  looking  for  its  visual  target.  The  rightmost  image  in  Figure  233 
shows  the  interior  of  the  copy  room  immediately  upon  entry,  and  with  no  sign  of  the  red  object,  the 
robot  eventually  stops  at  the  blue  icon  depicted  in  Figure  234a  (Arkin,  Collins,  &  Endo,  1999). 


Figure  233.  The  Pioneer  AT  test  surrogate  enters  the  hallway  (far  left),  traverses  past  Room  360, 
turns  towards  the  copy  room  (center),  enters  the  copy  room,  and  perceives  the  interior  as  devoid  of 
the  red  target  object  (far  right)  (images  adapted  from  Arkin,  Collins,  &  Endo,  1999). 

The  trial  was  then  repeated  with  the  red  target  object  prepositioned  in  the  copy  room.  The  mission 
was  initiated  from  the  same  starting  point  as  before  as  shown  in  Figure  234b,  with  the  robot 
traversing  the  hallway  past  Room  360  and  turning  in  towards  the  copy  room.  As  shown  in  Figure 
235,  the  red  target  object  was  immediately  detected  by  the  on-board  Cognachrome  vision  system 
developed  by  Newton  Research  Labs,  Renton,  WA  (Arkin,  Collins,  &  Endo,  1999),  causing  the  robot 
to  stop  prior  to  room  entry  (see  again  Figure  234b).  If  the  target  had  not  been  detected,  the  robot 
would  have  entered  the  room,  continuing  to  search,  as  in  the  previous  example  (Figure  233). 
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Figure  234.  a)  The  Pioneer  AT  initiates  the  first  remote-room-search  trial  at  lower  left,  then  enters 
the  copy  room  at  lower  right  after  not  finding  the  red  target  object,  b)  The  robot  initiates  the  second 
trial  as  before  at  lower  left,  but  stops  short  of  the  copy  room  upon  detecting  the  red  target  object 
(adapted  from  Arkin,  Collins,  &  Endo,  1999). 

In  July  and  October  of  1999,  these  preliminary  test  results  were  further  validated  at  the  Fort  Sam 
Houston  TMR  test  site  in  San  Antonio,  TX  (Arkin,  Collins,  &  Endo,  1999).  In  the  first  round  of  tests, 
the  robot  explored  a  corridor  inside  an  abandoned  hospital,  looking  for  an  open  room  that  potentially 
contained  a  biohazard,  with  a  priori  knowledge  of  the  approximate  room  location.  In  the  second 
round,  the  robot  had  to  visually  detect  a  wall-mounted  biohazard  sign  with  no  prior  knowledge  of  its 
location  in  order  to  locate  the  room  to  be  searched.  Eight  trials  were  run  under  the  first  scenario,  out 
of  which  the  robot  successfully  completed  the  mission  five  times,  with  five  completions  out  of  12 
trials  for  the  second  (Collins,  Arkin,  Cramer,  &  Endo,  2000). 


Figure  235.  The  Pioneer  AT  test  surrogate  enters  the  hallway  (far  left),  traverses  past  Room  360, 
turns  towards  the  copy  room  (center),  and  comes  to  a  halt  upon  detecting  the  target  object  in  the 
copy  room  (far  right)  (images  adapted  from  Arkin,  Collins,  &  Endo,  1999). 

The  DARPA  Phase  I  BAA  98-08  Part  B  -  System  Design  performers  included  the  below 
organizations  (Gage,  2000): 

•  Science  Applications  International  Corporation 

•  Draper  Lab 

•  Raytheon  Corporation 

The  following  BAA  97-20  Phase  //performers  were  the  Jet  Propulsion  Lab  (project  lead),  Carnegie 
Mellon  University  (navigation),  IS  Robotics  (mobility),  Oak  Ridge  National  Laboratory  (group 
behaviors),  and  the  University  of  Southern  California  (operator  control  unit).  The  project  focus  by 
this  point  was  to  develop  intelligent,  autonomous  navigation  for  small  UGVs  using  the  IS  Robotics 
Urbie  mobile  platform  (CMU,  2016a). 
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As  summarized  by  Doug  Gage  (2000),  the  technical  goal  was: 

“Development  of  a  system  of  robots  capable  of  operator  tasked  and 
monitored  perception-based  autonomous  mobility  in  diverse  unstructured 
environments  that  can  fit  in  a  rucksack  and  be  employed  in  coordinated 
groups  as  a  tool  for  the  dismounted  warfighter.” 

Autonomous  robotic  stair  climbing  was  viewed  as  a  critical  behavior  required  for  reconnaissance 
and/or  search  and  rescue  missions  in  urban  environments.  In  support  of  this  need,  the  NASA  Jet 
Propulsion  Laboratory  (JPL)  developed  two  estimation  and  control  algorithms  that  significantly 
increased  the  speed  and  effectiveness  of  autonomous  stair  climbing  (Helmick  et  al.,  2002): 

•  A  Kalman  filter  that  fused  visual  and  lidar  data  with  inertial  measurements  for  improved 
vehicle-attitude  estimates  at  a  high  rate 

•  A  physics-based  controller  that  minimized  heading  error  and  maximized  effective  forward 
velocity  during  stair  climbing 

Experimental  results  using  the  iRobot  PackBot  shown  in  Figure  236  validated  the  improved 
performance  of  this  approach  over  previous  methods  (Helmick  et  al.,  2002). 


Figure  236.  a)  The  JPL  stair-climbing  algorithms  were  evaluated  on  an  early  version  of  the  iRobot 
PackBot  outfitted  with  a  pair  of  video  cameras  and  a  2D  Sick  LMS-200  lidar.  b)  JPL  edge  detection 
algorithms  applied  to  single-camera  video  images  estimated  heading  0  and  center  position  dL/dR  at 
approximately  4  Hz,  which  allowed  differential  steering  to  keep  the  robot  on  heading  and  centered 
(adapted  from  Helmick,  Roumeliotis,  McHenry,  &  Matthies,  2002). 

Carnegie  Mellon  University  (CMU)  developed  monocular  visual  servoing  as  a  supervised 
autonomous  driving  mode  for  the  Urbie  robot,  wherein  the  remote  operator  would  designate  an  item 
of  interest  (such  as  a  door  or  stairway)  for  the  robot  to  approach  (CMU,  2016a).  By  servoing  on  the 
image  of  the  selected  target,  the  robot  could  autonomously  execute  the  driving  task  without  human 
assistance  (CMU,  2016b).  An  omnidirectional  camera  developed  by  Professor  Shree  Nayar  at 
Columbia  University  allowed  360-degree  target  selection  in  the  surrounding  environment  (Nayar, 
1997).  Technical  issues  included  the  selection  of  suitable  visual  templates  to  track,  seamless 
detection  and  recovery  in  the  event  of  loss  of  track,  and  integration  with  other  behaviors  such  as 
obstacle  avoidance  (CMU,  2016a). 
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The  ultimate  TMR  goal  of  implementing  intelligent  behaviors  on  the  PackBot  was  unfortunately 
cut  short  by  the  9/1 1  terrorist  attack  on  the  World  Trade  Center,  after  which  Lieutenant  Colonel 
Blitch  deployed  to  Iraq  with  several  teleoperated  program  assets.  As  the  program  closed  down,  Bart 
Everett,  Robin  Laird,  and  Cliff  Hudson  convinced  DARPA  to  transition  relevant  TMR  contracts  to 
the  Center,  and  subsequently  cherry  picked  promising  autonomy  technology  via  the  Technology 
Transfer  program.  That  same  afternoon,  they  visited  Jeff  Bradel  and  a  few  members  of  his  staff  at  the 
nearby  Office  of  Naval  Research  to  try  and  set  up  a  similar  arrangement  with  ONR  30.  (See  further 
Technology  Transfer  discussion  under  Project  Summaries,  Volume  2.) 

Man-Portable  Robotic  System  (1 999-2012) 

In  1999,  under  the  NAVSEA-funded  Man-Portable  Robotics  System  (MPRS)  project,  SSC  San 
Diego  developed  a  small  UGV  for  use  by  Army  engineers  in  tunnel,  sewer,  cave,  and  urban-structure 
reconnaissance.  A  small  65-pound  tracked  vehicle  based  upon  the  chassis  and  running  gear  of  the 
Foster-Miller  Tactical  Adjustable  Robot  (TAR),  the  URBOT  (Urban  Robot)  was  fully  invertible  in 
that  it  could  flip  over  and  continue  to  operate.  Early  URBOTs  were  teleoperated  from  a  wearable 
operator  control  unit  (OCU),  allowing  the  warfighter  to  drive  the  small  vehicle  into  high-risk  areas 
while  receiving  video  feedback  to  assess  the  situation  before  human  entry.  Initial  prototypes  were 
used  in  several  U.S.  Army  experiments  at  Fort  Leonard  Wood,  MO,  Fort  Drum,  NY  (Figure  237a), 
and  Fort  Polk,  LA  (Figure  237b). 


Figure  237.  a)  Soldiers  prepare  to  lower  an  URBOT  down  a  manhole  at  the  Fort  Drum  Military 
Operations  in  Urban  Terrain  (MOUT)  site  with  specially  designed  tackle  that  facilitated  post-mission 
asset  recovery,  b)  The  URBOT  is  about  to  enter  this  underground  culvert  on  a  counter-IED  training 
mission  at  Fort  Polk,  LA. 

The  front-mounted  adjustable  "snout,"  which  housed  various  sensors  and  illuminators  for 
situational  awareness,  could  be  tilted  ±90  degrees  to  look  up  or  down.  The  sensor  package  employed 
both  forward-looking  infrared  (FLIR)  and  low-light  cameras,  their  composite-video  outputs  fused 
into  a  single  image  through  a  "fader"  card.  By  fading  between  the  FLIR  and  low-light  camera,  the 
remote  operator  could  adjust  the  fusion  ratio  to  extract  the  maximum  amount  of  information  from  the 
combined  video  imagery.  The  result  was  presented  to  the  operator  on  a  small  ruggedized  hand-held 
display  as  shown  in  Figure  238. 
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Figure  238.  Bart  Everett  (center)  briefs  DARPA  Director  Dr.  Tony  Tether  (grey  suit)  on  the  Center’s 
unmanned  systems  projects,  as  Mike  Bruch  (lower  left)  operates  the  Urban  Robot  (not  shown)  using 
the  drive  pendent  (right  hand)  and  video  display  (left  hand)  of  the  custom-designed  back-packable 
controller,  circa  summer  2001 . 

Not  counting  the  initial  prototype  shown  in  Figure  237,  five  second-generation  URBOTS  were 
ultimately  constructed  by  the  Robotic  Systems  Branch,  the  first  of  which  is  shown  in  Figure  239a. 
Additional  upgrades  in  the  form  of  improved  electronics  and  heavy  duty  tracks  supplied  by  Foster- 
Miller  were  made  to  the  next  four  units  to  significantly  improve  reliability  and  performance  (Figure 
239b).  As  discussed  in  the  remainder  of  this  section,  these  rugged  and  reliable  systems  served  in 
multiple  roles  over  the  next  several  years  in  support  of  continued  development,  user  test  and 
evaluation  exercises,  crisis  response,  and  in-theater  deployment. 


Figure  239.  a)  And  early  second-generation  URBOT outfitted  with  test  equipment  to  evaluate  the 
performance  of  the  drive-motor  H-bridge  controllers  under  load,  b)  These  last  four  URBOTS  were 
retrofitted  with  the  much  improved  heavy-duty  Foster-Miller  tracks  developed  for  their  Talon  robot. 
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At  the  request  of  the  RS  JPO,  Bart  Everett,  Robin  Laird,  and  Mike  Bruch  supported  the  Center  for 
Robotic  Assisted  Search  and  Rescue  (CRASAR)  with  three  URBOTS  to  evaluate  structural  damage 
at  the  World  Trade  Center  site  after  the  terrorist  attack  on  1 1  September  2001  (Figure  240).  Other 
robotic  organizations  supporting  CRASAR  (Foster-Miller,  iRobot,  and  the  University  of  South 
Florida)  used  smaller  vehicles,  primarily  the  Foster-Miller  SOLEM  and  Inukton  Vari-Track,  to  better 
penetrate  the  rubble  pile  and  identify  areas  where  victims  were  located  (Mullens,  2001).  Following 
this  week-long  evolution,  attention  returned  to  the  tactical  applications  for  which  the  URBOT  was 
originally  designed. 


Figure  240.  a)  Robin  Laird  controls  the  Foster-Miller  Special  Operations  Lemming  (SOLEM)  robot 
from  its  compact  self-contained  operator  control  unit,  b)  Mike  Bruch  (right)  controls  one  of  three 
URBOTs  while  Bart  Everett  looks  on,  with  the  remains  of  WTC  Tower  Two  in  background. 

In  April  2002,  two  URBOTs  and  two  Inukton  Variable  Geometry  Tracked  Vehicles  (VGTVs)  were 
deployed  to  Afghanistan  with  U.S.  Navy  EOD  forces  to  assist  in  clearing  buildings  and  caves  (Figure 
241).  These  assets  were  part  of  the  Robotic  Systems  Pool,  funded  by  the  OSD  Joint  Robotics 
Program  and  managed  by  SSC  San  Diego,  as  later  discussed  in  the  Project  Summaries  section. 
Volume  2  (Mullens,  2002a).  The  robots  were  used  by  Navy  EOD  personnel  for  the  remote  inspection 
of  mines,  booby  traps,  and  what  would  soon  become  known  as  improvised  explosive  devices  (IEDs). 
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Figure  241 .  Lieutenant  (j.g.)  Kevin  Childre  loads  a  pair  each  of  URBOTs  and  Inukton  VGTVs 
provided  to  EOD  Mobile  Unit  3  (EODMU3)  by  SSC  San  Diego  for  in-theater  deployment  on  1  April 
2002. 

In  June  that  same  year.  Colonel  Bruce  Jette,  U.S.  Army,  led  a  Robotic  Tiger  Team  on  a  90-day 
quick-response  technology  insertion  in  Afghanistan  to  provide  commercial  man-portable  robots  to 
warfighters  engaged  in  life-threatening  roles.52  Made  up  of  representatives  from  iRobot,  Exponent , 
University  of  Southern  California’s  Institute  for  Creative  Technologies  (Figure  242),  and  active  duty 
Army  personnel,  the  goals  were:  1)  to  improve  mission  capability  while  decreasing  risk;  2)  assess  the 
full  range  of  tactics,  techniques,  and  procedures  (TTPs)  for  small  robots  in  an  operational 
environment;  and  3)  examine  the  entire  process  involved  in  rapid  integration  of  robotic  systems 
(REF,  2015). 


Figure  242.  Ted  Hromadka  of  Exponent  (left),  Tom  Frost  of  iRobot  (center),  and  Dave  Hendrie  of  the 
USC  Institute  of  Creative  Technologies  (right)  en  route  to  Afghanistan  in  support  of  the  Rapid 
Equipping  Force  on  28  June  2002  (image  courtesy  iRobot  Corporation). 


52  The  official  team  designation  was  “Rapid  Integration  of  Robotic  Systems”  (REF,  2015). 
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Based  out  of  Bagram  Air  Base,  the  team  conducted  missions  in  Kandahar  (Figure  243),  Gardez, 
and  Kwost  to  mitigate  combat  casualties  from  booby  traps  and  grenade  blowback  while  searching 
and  clearing  caves  (REF,  2015).  The  resulting  successes  led  Colonel  Jette  to  recommend  to  the  Vice 
Chief  of  Staff  that  the  Army  continue  to  pursue  spiral  fielding  of  robots,  and  establish  an  equipping 
strike  force,  a  supporting  sustainment  process,  and  a  supporting  technology  mining  team.  The  Rapid 
Equipping  Force  (REF),  headquartered  at  Fort  Belvoir,  VA,  was  subsequently  stood  up  under  the 
Army  G-3/5/7  in  late  2002  to  support  warfighters  in  Operation  Enduring  Freedom  using  deployed 
teams  of  subject  matter  experts  (REF,  2015). 53 


Figure  243.  a)  An  Iraqi  interpreter  observes  an  iRobot  PackBot  exiting  a  suspect  bunker  in 
Kandahar,  b)  Approximately  65  pounds,  the  PackBot  could  be  portaged  to  the  start  of  a  mission,  as 
for  example  exploring  this  cave  in  Afghanistan.  Note  the  wearable  M7  Operator  Control  Unit 
(courtesy  iRobot  Corporation). 

Also  in  2002,  a  Chemical,  Biological,  Radiological,  and  Nuclear  (CBRN)  plug-and-play  sensor 
payload  was  developed  for  the  URBOT  at  the  request  of  the  U.S.  Army  Chemical  School  at  Fort 
Leonard  Wood,  MO  (Figure  244a).  Towards  the  end  of  this  effort,  a  Limited  Objective  Experiment 
(LOE)  was  conducted  in  February  2003  at  SSC  San  Diego  to  test  the  payload  and  its  standard-issue 
sensors:  the  MultiRAE  Plus  gas  sensor,  the  Joint  Chemical  Agent  Detector,  and  the  AN/UDR-13 
Radiac  (Figure  244b).  The  payload  was  subsequently  adapted  for  use  on  the  Mesa  Engineering 
Matilda  robot  and  demonstrated  by  Major  Todd  Cline  of  the  RS  JPO  in  Kuwait  to  the  Combined 
Joint  Task  Force  -  Consequence  Management. 


53  On  30  January  2014,  the  Army  declared  the  REF  an  enduring  capability,  which  was  subsequently  realigned  under 
the  U.S.  Army  Training  and  Doctrine  Command. 
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Figure  244.  a)  The  prototype  chemical  sensor  payload  is  shown  here  on  the  URBOT,  circa  2002. 
b)  Interior  view  of  the  sensor  payload  showing  the  MultiRAE  Plus  gas  sensor  (bottom  left),  the 
AN/UDR-13  Radiac  (upper  left),  and  the  Joint  Chemical  Agent  Detector  (upper  right). 

As  a  result  of  their  efforts  to  expedite  this  development  in  preparation  for  Operation  Iraqi 
Freedom ,  Michael  Bruch,  Aaron  Burmeister,  Jason  Lum,  and  Bart  Everett  were  each  presented 
Brigadier  General  Nilo's  “Coin  of  Excellence”  by  Chemical  Regimental  Command  Sergeant  Major 
CSM  Peter  Hiltner  (Mullens,  2003).  A  third-generation  version  of  this  payload,  the  Chemical  and 
Hazardous  Avoidance  Robotic  System  ( CHARS)  payload  was  ported  to  four  iRobot  PackBot 
Explorers  provided  via  the  RS  JPO  to  the  82nd  Airborne  Division  during  Operation  Iraqi  Freedom  in 
2003  (Mullens,  2004). 


Figure  245.  One  of  four  iRobot  PackBot  Explorers  equipped  with  the  CHARS  payload.  Note  iRobot’s 
fiber-optic  spooler  for  improved  communications  when  entering  caves  and  bunkers. 

Shown  in  Figure  246a,  an  experimental  tandem  URBOT  configuration  facilitated  climbing 
otherwise  insurmountable  obstacles,  but  suffered  from  impaired  turning  performance,  with  the  ability 
to  spin  in  place  eliminated  altogether.  An  improved  linkage  designed  by  lead  mechanical  engineer 
Aaron  Burmeister  allowed  the  two  vehicles  to  decouple  for  independent  maneuvering,  then  reconnect 
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when  desired  (Figure  246b).  There  was  some  interest  at  the  time  in  using  the  slave  URBOT  of  a 
tandem  pair  as  a  RF  communications  relay,  decoupling  it  to  serve  as  a  static  communications  node 
while  the  lead  vehicle  continued  to  advance.  This  application  was  never  pursued,  however,  given  the 
far  more  practical  approach  of  dropping  a  small  RF  repeater  brick,  as  further  discussed  in  several 
later  sections,  Volume  2. 


Figure  246.  a)  A  tandem  URBOT  configuration  was  evaluated  for  potentially  improved  mobility  over 
rugged  terrain,  b)  An  improved  second-generation  design  featured  a  flexible  linkage  that  could  be 
automatically  decoupled  and  later  reconnected  on  demand.  Note  GPS  antenna  on  lead  vehicle. 
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APPENDIX  A  -  NAVSEA  POLICY  ON  AUTOMATION  AND  ROBOTICS 


DEPARTMENT  OF  THE  NAVY 


2020 

Ser  90G/63 

11  May  1984 


N  REPLY  REFER  TO 


Fran:  Cbmmander,  Naval  Sea  Systems  Command 
lb:  Distribution 


Subj:  NAVSEA  POLICY  RELATIVE  TO  INTELLIGENT  MACHINE  AUTOMATION  AND 
ROBOTICS 


1*  Intelligent  Machine  Autonation  is  a  new  area  of  advanced  technology  being 
extensively  researched#  developed  and  applied  within  industry.  As  a 
multi-billion  dollar  claimant  of  the  industrial  resources  of  the  nation#  the 
Navy  must  capitalize  on  the  tremendous  potential  of  this  new  technology  in  the 
manufacture#  maintenance  and  operation  of  ship  and  coiribat  systems  when  it  is 
cost  effective  to  do  so.  In  this  regard#  an  inportant  goal  of  this  Command  is 
to  ensure  that  the  Navy  utilizes  intelligent  systems  to  reduce  total 
life-cycle  costs#  guarantee  quality#  improve  readiness#  extend  endurance#  free 
human  assets  for  higher-order  functions#  and  enhance  the  attractiveness  of 
shipboard  life  for  Naval  personnel. 

2.  It  is  recognized  that  all  parts  of  such  a  comprehensive  goal  cannot  be 
achieved  simultaneously#  tut  because  they  are  interrelated  they  should  be 
pursued  on  an  integrated  basis.  A  NAVSEA  Robotic©  Program  Plan  (under 
development)  will  set  forth  the  planned  actions  to  achieve  this  integration. 

In  the  interim#  addressees  are  encouraged  to  investigate  ways  to  utilize  this 
new  technology  to  inprove  quality  and  proto tivity  in  their  functional  areas. 

3.  She  Robotics  Program  Plan#  Which  will  reflect  the  deliberations  and  the 
earlier  independent  work  of  the  members  of  the  NAVSEA  fobotics  Council#  will 
fully  develop  the  following  approach:  establish  an  understanding  and  Command 
awareness  of  the  state-of-the-art  and  the  current  prognosis  for 

robot ics/lntelligent  Machine  Automation  technology;  accumulate  and  organize  a 
readily  accessible  data  base;  critically  and  constructively  review  all  ongoing 
projects  and  ensure  that  continuing  efforts  in  those  projects  are  addressed  to 
accomplishments  consistent  with  the  major  thrusts  comprising  the  Command  goal; 
identify  Navy  needs  for  robotic  applications;  natch  those  needs  to  technology 
requirements;  address  throuch  research  and  development  efforts  those 
technology  requirements  that  are  not  being  addressed  by  the  private  sector? 
share  technology  across  as  broad  a  commercial  and  Navy  spectrum  as  is 
feasible;  develop  and  demonstrate  prototype  applications?  conduct  cost  benefit 
analyses  on  all  promising  prototypes?  select  and  implement  winners?  and# 
sponsor  an  appreciation  of  the  robotic  power  of  proven  prototypes  throughout 
the  Navy. 
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4.  lb  facilitate  the  attainment  of  this  goal,  LCDR  Bart  Everett  (SEA  90G)  is 
designated  as  my  Special  Assistant  for  Robotics.  Be  will  be  responsible  for 
developing  and  executing  the  Robotics  Program  Plan;  acquiring,  synthesizing 
and  disseminating  robotics  technological  information;  conducting  studies, 
convening  workshops  and  attending  industrial  and  academic  conferences  and 
demonstrations;  and  participating  as  a  counselor /adviBor  to  NAVSEA  in 
exploratory  and  decision  briefings  related  to  robotics  applications  in  ship 
and  weapon  acquisitions,  maintenance  and  operational  programs. 

5.  lb  minimize  redundancy  in  development  efforts,  to  ensure  the  compatibility 
of  independently  developed  systems,  to  avoid  the  risks  of  inappropriately 
assigned  or  ill-conceived  applications,  and  to  ensure  that  progress  is  made  on 
the  broad  spectrum  of  the  Robotics  Program  Plan  (When  promulgated),  all  NAVSEA 
robotics  efforts  shall  be  coordinated  with  SEA  90G. 

6.  The  NAVSEA  Rcbotics/Artificial  Intelligence  Eata  Base  has  been  established 
at  the  Naval  Oceans  Systems  Center  to  provide  the  mechanism  for  descriptive 
and  status  information  on  all  ongoing  and  planned  robotics  projects. 

Individual  codes  initiating  programs  in  robotics  are  tasked  to  become  aware  of 
and  take  advantage  of  ongoing  and  planned  efforts,  and  to  ensure  that  new 
initiatives  are  properly  scoped  to  provide  the  maximum  advantage  to  the 
Command. 

7.  All  of  the  foregoing  is  intended  to  provide  a  Command  atmosphere  of 
acceptance  for  initiatives  in  this  inportant  new  technology.  I  call  upon  all 
of  yew  to  examine  the  opportunities  that  robotics  offers  to  your  functional 
responsibilities,  and  to  consider  the  allocation  of  some  of  your  resources  in 
appropriate  projects  as  an  investment  in  the  Navy's  future. 

ft)  • 

E.  B.  FOWLER 


Distribution: 
SEA  003 
SEA  05 
SEA  06 
SEA  07 
SEA  90 
SEA  91 
SEA  92 
ms  309 
MS  400 
PDS  350 
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APPENDIX  B  -  WHITE  PAPER  PRESENTED  TO  ONR/ONT 


WHITE  PAPER 


ROBOTICS  TECHNOLOGY 

AREAS  OF  NEEDED  RESEARCH  AND  DEVELOPMENT 


Prepared  by 

LCDR  H.  R.  Everett,  DSN 
OFFICE  OF  ROBOTICS  AND  AUTONOMOUS  SYSTEMS 
NAVAL  SEA  SYSTEMS  COMMAND 
WASHINGTON,  DC  20362 


Presented  to 

OFFICE  OF  NAVAL  RESEARCH 
OFFICE  OF  NAVAL  TECHNOLOGY 


6  September  1985 
Ser  90G/119 
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KEY  ISSUE 


-  What  is  meant  by  the  term  "robotics  technology*? 

POINTS 

-  Attempts  to  define  a  "robot"  have  been  made  by  .  many 
organizations,  including  the  Robotics  Industries  Association  and 
the  Naval  Air  Systems  Command*  None  of  these  definitions  are 
adequate  for  Navy-wide  use  in  both  the  industrial  and  non¬ 
industrial  sense* 

—  Industry  definitions  lack  operational  (non-industrial) 
orientation 

—  Service  definitions  do  not  adequately  address  military 
manufacturing  and  repair  (industrial)  applications 

-  The  Joint  Technology  Panel  for  Robotics  (JTPR) ,  on  behalf  of 
the  Joint  Directors  of  Laboratories  (JDL)  has  established  the 
following  definition  of  robotics: 

"A  system  incorporating  a  computer  controller  to  provide 
autonomy  and  reprogrammability ,  which  employs  an  end-effector  of 
some  type  (manipulator  arm  or  mobile  platform) ,  which  exhibits 
flexibility  in  the  roles  which  it  can  perform  or  the  equipment 
with  which  it  interfaces,  and  which  performs  tasks  of  a 
complexity  level  that  previously  required  human  control*" 

-  The  issue,  however,  is  not  really  the  definition  of  a  "robot", 
but  rather  what  is  meant  by  "robotic  technology*"  The  field  of 
robotics  (assume  for  now  the  JTPR  definition)  is  supported  by  the 
disciplines  of  mathematics,  computer  science  (to  include 
artificial  intelligence),  mechanical  and  electrical  engineering, 
materials,  physics,  psychology,  and  anatomy.  These  supporting 
disciplines  required  to  construct  a  mechanical  system,  endow  it 
with  intelligence,  and  provide  the  necessary  sensor  data  upon 
which  to  act,  can  collectively  be  termed  robotics  technology* 

CONCLUSIONS 

-  There  is  no  universally  accepted  definition  of  a  "robot",  but 
that  put  forth  by  the  JTPR  appears  the  most  appropriate  for  this 
discussion*  Revision  of  the  JTPR  definition  will  add  confusion, 
but  will  not  necessarily  gain  acceptance. 

-  All  the  supporting  disciplines  contributing  to  the  successful 
fielding  of  ah  intelligent  system  as  so  defined  can  be  considered 
"robotic  technology"  when  thus  employed* 

RECOMMENDATION 

-  Use  the  JTPR  definition  of  robotics  to  facilitate  determination 
of  technologies  that  should  be  pursued  in  support  of  robotics 
system  development* 
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ROBOTICS  TECHNOLOGY  RESEARCH  AND  DEVELOPMENT  NEEDS 


B&ffiKi SfigfflffiL 

For  the,  increasing  number  of  identified  potential  Navy 
applications#  there  are  known  deficiencies  in  the  supporting 
technologies  that  will  impede#  if  not  preclude#  successful 
implementation  of  robotic  solutions.  These  can  be  subdivided  into 
the  two  general  categories  of  "Industrial"  and  "Non-industrial"# 
and  are  summarized  below. 


an  pniqpb  lacaimiAit  m 

For  the  most  part  these  needs  reflect  the  fundamental 
differences  between  conventional  high-volume  assembly-line 
scenarios  which  pervade  throughout  industry#  and  the  very  low 
volume#  unstructured  environments  of  Navy  applications  in 
manufacturing#  but  even  more  importantly#  maintenance  and  repair. 
It  is  this  latter  area  where  the  major  impact  of  robotics  on  the 
Navy  is  predicted  to  occur#  and  this  arena  has  been  virtually 
untouched  by  industrial  developments.  Examples  of  needed 
research  and  developments 

On-line  Programming  Techniques  -  Acceptable  methods  must  be 
developed  to  allow  faster  programming  for  low  volume 
applications.  Conventional  teach  pendants  employed  by  industry 
are  impractical  in  Navy  scenarios.  Options  include  laser-based 
target  designation  systems,  six  degree-of-f reedom  joysticks# 
voice  input,  etc. 

Off-line  Programming  Techniques  -  Practical  methods  must  be 
devised  to  provide  three  dimensional  data  describing  part 
geometries  for  use  in  generating  robot  motion  programs.  This 
requires  interfacing  with  and  augmenting  existing  and  future 
Computer  Aided  Design  (CAD)  systems#  and  the  development  of 
volumetric  digitizing  techniques  and  sensors.  In  addition# 
process  control  information  depicting  parameter  values  and  the 
sequence  of  operations  must  be  easily  represented  in  the  design 
database  to  allow  intelligent  robotic  systems  to  address  low 
volume#  unstructured  scenarios  typical  of  Navy  applications. 

Path  Planning  for  Industrial  Robots  -  Appropriate  algorithms  must 
be  developed  to  automatically  generate  the  optimum  manipulator 
and  end-effector  responses  from  the  geometric  and  process  control 
data  discussed  above. 

Collision  Avoidance  -  Specialized  algorithms  must  be  devised  to 
ensure  that  robot  motion  trajectories  and  process  sequences 
calculated  in  an  offline  mode  do  not  create  hazardous  situations 
in  terms  of  damage  to  the  workpiece#  the  equipment#  or  operating 
personnel.  This  requires  extensive  dynamic  modeling  of  the 
robotic  systems#  workpieces,  and  associated  environments. 
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Sensors  to  Support  Collision  Avoidance  -  Three  dimensional 
imaging  sensors  are  required  to  ascertain  part  location  and 
orientation  for  input  to  the  collision  avoidance  software 
routines,  as  well  as  to  identify  discrepancies  between  expected 
and  actual  conditions* 

Real  Time  Process  Control  Sensors  and  Algorithms  -  Automatic  and 
adaptive  process  control  is  essential  if  robotic  systems  are  to 
be  employed  in  Navy  industrial  scenarios,  due  to  the  unstructured 
and  changing  working  environments.  Research  issues  include  but 
are  not  limited  to  weld  pool  imaging  systems,  infrared 
thermography,  paint  thickness  guaging,  surface  cleanliness 
sensors,  non-contact  measurement  techniques,  seam  tracking 
systems,  weld  process  control  strategies.  Typical  applications 
include  surface  preparation  and  coating,  gas  metal  arc  welding, 
laser  metalworking,  application  of  flame  sprayed  coatings, 
grinding  and  polishing,  non-destructive  testing,  etc. 

Dynamic  Control  Techniques  -  This  is  a  critical  research  issue 
needed  to  support  the  design  of  large  robotic  systems  capabl^  of 
dealing  with  massive  workpieces  as  encountered  in  ship  and 
weapons  system  manufacturing  and  repair  scenarios.  Conventional 
industrial  robots  have  in  comparison  rather  limited  working 
envelopes.  They  can  therefore  assign  constant  values  to  control 
system  parameters  and  mechanical  properties  such  as  moments  of 
inertia,  static  and  dynamic  frictional  forces,  etc.  In  reality, 
however,  these  entities  are  not  fixed  values  but  functions  of 
manipulator  and  end-effector  position,  velocity,  and 
acceleration,  and  further  affected  by  changing  payloads. 
Accuracy,  repeatability,  and  system  response  degrade  measurably 
as  real  world  conditions  vary  from  ideal  assumptions,  and  large 
systems  will  require  real  time  calculation  of  servo  control 
mechanism  transfer  functions  (i.e.,  gains,  damping  coefficients, 
etc.)  in  order  to  compensate# 

Computer  Simulation  of  Robotic  Devices  -  Much  work  is  needed  in 
this  area  to  provide  generic  tools  needed  for  off-line 
programming,  collision  avoidance,  path  planning,  and  dynamic 
control  research. 

Generic  Rule-Based  Architectures  -  The  development  of  a  generic 
system  architecture  for  networking  a  modular  collection  of  expert 
systems  with  the  appropriate  modular  sensor  and  controller 
subsystems  is  viewed  as  necessary  and  desirable  for  complex  Navy 
applications.  Such  an  architecture  would  provide  for  *  inherent 
standardization  as  well  as  allow  for  evolutionary  system  upgrades 
in  response  to  componentry  improvements.  The  rule-based  expert 
systems  address  the  CAD  interface,  path  planning,  collision 
avoidance,  and  scheduling  functions  discussed  above,  and  could  be 
modified  through  rule  changes  to  accomodate  different  system 
applications,  without  extensive  redesign. 

Expert  System  Development  -  Generic  research  in  expert  system 
development  is  mandatory  for  providing  the  required  system 
intelligence  to  allow  conventional  robotic  systems  to  address  in 


a  practical  fashion  complex  Navy  needs. 

Ship  Motion  Effects  Research  is  needed  to  investigate  the 
effects  of  ship  motion  on  robot  dynamics  and  equipment  life# 


NON-INDUSTRIAL  NEEDS 


The  following  research  and  development  needs  are  required  to 
support  operational  applications  of  robotics,  embodied  for  the 
most  part  in  mobile  systems.  Initial  emphasis  in  prototype 
development  will  address  hazardous  operations  (EOD,  NBC 
scenarios)  and  performance  of  tasks  for  which  man  is  incapable. 
As  advances  are  made  in  the  supporting  technologies,  there  will 
be  a  natural  trend  from  teleoperated  to  semi-autonomous  and 
autonomous  systems. 

Collision  *  Avoidance  for  Mobile  Robots  -  Regardless  of  the 
application,  an  essential  technological  need  for  any  system 
involving  mobility  will  be  the  capability  to  avoid  impact  with 
surrounding  objects.  The  problems  associated  with  this  need  are 
two-folds  1)  the  acquisition  of  high  resolution  geometric  data 
describing  the  environment,  and  2)  the  computational  resources 
needed  to  interpret  that  data. 

Sensors  to  Support  Collision  Avoidance  -  The  acquisition  of 
geometric  data  requires  the  development  of  high  resolution,  low 
cost  non-contact  ranging  systems  capable  of  real  time  operation. 
Ultrasonic  ranging  systems  have  served  in  this  capacity,  but 
suffer  from  problems  associated  with  extremely  poor  angular 
resolution,  temperature  dependence,  specular  reflection, 
interference  from  adjacent  units,  and  the  relatively  slow  speed 
of  sound  in  air.  Conventional  laser  rangefinders  are 
prohibitively  expensive  in  terms  of  initial  costs,  physical  size, 
and  energy  requirements.  Practical  units  must  be  employed  in 
sufficient  numbers  to  rapidly  acquire  geometric  data  for  use  in 
modeling  the  robot's  surroundings,  to  support  decisions  on 
terrain  traversibility ,  and  to  address  environmental  •awareness1' 
in  general. 

Navigational  Planning  for  Mobile  Robots  -  Mobile  autonomous  and 
even  teleoperated  systems  must  be  capable  of  determining  their 
exact  location  as  well  as  their  orientation  at  that  location  in 
order  to  effectively  maneuver  to  a  desired  position,  to 
circumvent  known  obstructions  or  hazards,  or  to  avoid  detection. 
Secondly,  they  must  be  able  to  calculate  the  optimum  path  for 
traversing  from  their  current  location  to  the  goal,  a  task  which 
is  computationally  exhausting. 

Sensors  to  Support  Navigational  Planning  -  The  task  of 
ascertaining  position  and  orientation  will  require  the 
development  of  low  cost,  accurate,  and  reliable  sensors  and/or 
navigational  aids  currently  unavailable. 
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Computational  Resources  -  Improved  data  processing  techniques  as 
well  as  pipeline  and  parallel  processing  architectures  must  be 
developed  to  handle  the  massive  amounts  of  data,  calculations, 
and  symbolic  reasoning  needed  to  emulate  the  required  degree  of 
intelligence  for  even  the  most  primitive  of  systems.  This  is 
especially  critical  for  a  mobile  system  where  space  and  energy 
resources  are  at  a  premium. 

Application  Specific  Sensors  and  Controls  -  This  is  the  non¬ 
industrial  analogy  to  process  control  sensors  in  the  industrial 
sense.  For  a  given  functional  application  (firefighting,  sentry 
and  security  functions,  explosive  ordnance  disposal,  mine 
placement  and  neutralization,  undersea  search  and  recovery, 
airborne  sensor  platforms,  underwater  sensor  platforms,  weapons 
handling,  material  handling,  nuclear  maintenance,  containment, 
surveillance,  etc)  there  will  be  required  an  appropriate  sensor 
suite  and  associated  intelligence  to  effect  the  required  actions 
of  the  system. 

Motion  Effectors  -  Research  is  needed  to  further  develop  various 
types  of  motion  effectors  (tracked,  wheeled,  legged,  omni¬ 
directional)  for  optimal  maneuverability,  dexterity,  traction, 
etc. 

Energy  Sources  -  Mobile  systems  will  require  a  practical  onboard 
source  of  energy  to  support  drive  mechanisms,  actuators,  sensors, 
and  computational  resources. 

Man-Machine  Interface  -  Considerable  research  is  needed  in  this 
area  to  effectively  enhance  the  human  transfer  function  and  allow 
efficient  interaction  between  the  operator  and  the  complex 
teleoperated  and  semi-autonomous  systems  to  be  developed. 

Training  and  Self-Diagnostics  -  The  importance  of  this  area 
cannot  be  overemphasized.  Robotic  systems  of  the  future  will  by 
necessity  be  complex  in  nature,  and  not  well  understood  by  their 
users.  Substantial  gains  in  productivity,  quality,  or  safety 
could  be  easily  offset  by  problems  associated  with  operator 
training,  system  integration,  and  maintenance  and  repair.  It  is 
impractical  to  attempt  to  provide  the  skill  levels  needed  to 
support  such  equipment  through  conventional  means.  Such  action 
would  be  prohibitively  expensive,  and  even  if  theoretically 
possible  would  suffer  from  the  almost  certain  loss  of  highly 
trained  personnel  to  better  paying  jobs  in  industry.  Therefore, 
proposed  systems  must  be  fully  proficient  in  diagnosing  their  own 
problems..  Video  disk  technology  and  expert  systems  must  be 
developed  for  training  and  instruction  to  overcome  this  problem. 
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Ongoing  6.3  development  efforts  have  shown  the  requirement 
for  more  supportive  6.1  and  6.2  research.  The  examples  cited 
above  can  be  traced  to  specific  prototype  development  needs  in 
existing  NAVSEA  programs.  The  centralized  development  of  generic 
technology  in  response  to  these  issues  will  result  in  substantial 
cost  savings  to  the  Navy  through  avoidance  of  unwanted 
redundancy.  Additional  cost  savings  will  be  realized  through  the 
attainment  of  application  goals  otherwise  not  technologically 
feasible . 
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NC4(34) /cca 
18  May  1982 


Vice  Admiral  E.  B.  Fowler,  Jr,  USN 
Commander ,  Naval  Sea  Systems  Command 
Washington,  DC  20362 

Dear  Admiral  Fowler, 

I’m  writing  to  follow  up  on  our  conversation  of  30  April  1982  rela¬ 
tive  to  robotics.  As  you  may  recall,  I  noted  that  LCDR  Everett  of  the 
Naval  Engineering  Program  had  achieved  instant  nation-wide  media  attention 
through  his  robot ics-relatcd  thesis  research  and  that  my  observations 
indicated  that  LCDR  Everett  potentially  represents  a  real  "blue  suit" 
expert  in  a  field  that  is  about  to  explode  upon  the  U.S.  scene. 

I  have  discussed  further  the  subject  of  robotics  within  the  Navy 
with  LCDR  Everett  and  several  points  were  brought  forth.  LCDR  Everett  is 
very  concerned  about  efforts  to  ensure  that  some  naval  officers  are  being 
given  the  training  and  exposure  necessary  to  be  able  to  effectively  manage 
the  Navy’s  interests  with  regard  to  robotics.  He  anticipates  a  tremendous 
.impact  in  the  very  near  future  which  will  require  many  decisions  to  be 
made  as  to  what  robotics  can  do  in  our  shipyards,  what  would  be  imprac¬ 
tical,  and  which  systems  would  prove  cost  effective. 

io  be  prepared  for  this  situation  he  strongly  advocates  that  a  few 
individuals  be  identified  as  likely  candidates,  receive  appropriate  post¬ 
graduate  education,  and  then  serve  an  immediate  follow-on  tour  at  facili¬ 
ties  actively  involved  in  robotics  research,  such  as  NOSC,  San  Diego, 
lie  believes  that  only  through  this  active  association  after  formal 
education  can  an  individual  get  a  grasp  of  this  rapidly  growing  technology 
which  promises  to  radically  change  industrial  techniques  as  the  Navy  knows 
them  today.  To  expect  success  otherwise  invites  financial  disaster  as 
well  as  unreliable  productivity  as  the  systems  are  integrated  into  the 
present  industrial  environment. 

I  believe  that  LCDR  Everett  is  an  ideal  candidate  to  Implement  a 
program  to  investigate  and  prepare  for  this  upcoming  transition.  He  has 
an  intense  interest  in  the  subject  of  robotics,  and  has  built  four  work¬ 
ing  prototypes,  his  first  when  he  was  a  sophomore  in  high  school.  He 
began  a  systematic  approach  to  achieving  his  goal  of  becoming  an  expert 
in  the  field  by  acquiring  a  BS  in  Electrical  Engineering  at  Ccorgia  Tech, 
and  then  choosing  to  work  towards  an  MS  In  Mechanical  Engineering  here. 

He  has  read  extensively  outside  of  the  normal  program  of  study  in  order 
to  obtain  the  required  background  in  Computer  Science  while  maintaining 
an  overall  grade  average  of  3.®+ to  date.  His  current  prototype  has  won 
him  national  recognition  as  I’ve  noted  and  an  Invitation  to  join  Sigma 
Xi.  LCDR  Everett  intends  to  continue  his  personal  development  in  robotics 
even  after  departure  from  the  Naval  Postgraduate  School.  It  is  unlikely 
that  an  individual  could  be  found  who  possesses  the  required  background 
as  well  as  the  drive  and  personal  interest  of  LCDR  Everett  in  this 
complex  area. 
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NC4(34)/cca 
18  Kay  1982 


Of  the  many  ED  officers  I've  been  privileged  to  meet  and  observe 
while  assigned  to  NPS,  tCDR  Everett  comes  closest  to  representing  a 
"blue  suit"  technical  expert  in  a  rapidly  growing  technological  area* 
I  recommend  that  the  Navy  take  advantage  of  Everett1 a  expertise  in 
the  best  manner  possible  as  soon  as  possible* 

Very  respectfully , 

C.  &.  CAR&2TS0N 
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APPENDIX  D  -  NAVSEA  MEMO  TO  NMPC 


DEPARTMENT  OF  THE  NAVY 

NAVAL  SLA  SYSTEMS  COMMAND 
WASHINGTON  Of:  MWMlOl 


IN  ftliPLT  HLFtH  TQ 


2020 

Ser  90G/105 

From:  Commander,  Naval  Sea  Systems  Command 

To:  Commander,  Naval  Military  Personnel  Command 

Sub  j :  REASSIGNMENT  OF  LCDR  H  *  R,  EVERETT,  DSN  (250-7  8-4329/1440) 

Enel:  (1)  Captain  Garritson's  ltr  of  18  May  82 

(2)  NAVSEA  Integrated  Robotics  Program,  Annual  Report, 
FY-85. 


1-  In  recognition  of  LCDR  Everett*s  outstanding  performance  of 
duty  in  establishing  the  NAVSEA  Integrated  Robotics  Program 
during  his  extended  assignment  in  NAVSEA,  I  invite  your  attention 
to  the  following  issues  which  should  be  considered  in  addressing 
his  reassignment. 

a)  As  suggested  in  enclosure  (1) ,  the  most  appropriate 
assignment  for  LCDR  Everett  following  his  graduation  from  the 
Naval  Postgraduate  School  was  to  a  Navy  Laboratory  or  Center 
where  his  classroom  theoretical  training  could  be  honed  in  a 
"hands-on"  environment;  NOSC  was  noted  as  having  that  kind  of 
work  in  progress . 

b)  Immediate  needs  within  this  Command  at  the  time  made  it 
first  necessary  to  establish  a  NAVSEA  Robotics  Program  for 
purposes  of  technical  oversight  and  coordination*  LCDR  Everett 
was  selected  to  give  uniformed  presence  and  technically  advanced 
leadership  to  that  effort,  thus  foregoing  the  preferred 
assignment  to  NOSC, 

c)  The  assessment  and  organizational  planning  work  which 
necessitated  LCDR  Everett's  assignment  to  NAVSEA  is  essentially 
completed,  and  well  documented  in  enclosure  (2),  which  shows  that 
the  NAVSEA  activity  has  grown  from  a  few  independently  originated 
efforts  to  an  integrated  program  consisting  of  more  than  70 
projects . 

d)  The  program  noto  possesses  both  the  momentum  and  the 
infrastructure  to  sustain  itself  through  a  change  of  leadership? 
LCDR  Everett  can  therefore  be  reassigned  for  the  purpose  of 
enhancing  his  professional  technical  competence  through  direct 
association  with  technologically  demanding  projects.  The  research 
environment  at  NOSC  is  felt  to  be  the  most  appropriate  from  the 
standpoint  of  exposure  to  in-house  expertise  in  advanced  computer 
architectures,  artifical  intelligence,  and  autonomous  systems 
development. 
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2*  Accordingly,  it  is  my  desire  that  LCDR  Everett  be  assigned  to 
NOSC  now  for  planning  purposes,  recognizing  that  execution  of  the 
orders  may  have  to  be  delayed  until  next  fiscal  year* 
Anticipating  his  detachment  not  later  than  October  1986,  search 
must  begin  immediately  for  a  technically  qualified  relief 
(1440/1460  with  a  masters  degree  in  one  or  more  of  the  robotics 
component  technology  disciplines)*  It  is  permissable  to  gap  the 
billet  in  the  interim  in  the  interest  of  identifying  the  best 
qualified  candidate* 


Copy  to: 

SEA  90 
NOSC 
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APPENDIX  E  -  CENTER  AND  BRANCH  NAMES 


Date 

Center 

Branch 

Code 

Branch  Head 

1984 

NOSC 

Autonomous  Systems 

442 

Scott  Harmon 

1985 

NOSC 

Autonomous  Systems 

442 

Doug  Gage 

1986 

NOSC 

Autonomous  Systems 

442 

Doug  Gage 

1987 

NOSC 

Autonomous  Systems 

442 

Doug  Gage 

1988 

NOSC 

Advanced  Technology  Development 

535 

Ray  Glass 

1989 

NOSC 

Advanced  Technology  Development 

535 

Ray  Glass 

1990 

NOSC 

Advanced  Technology  Development 

535 

Steve  Martin 

1991 

NOSC 

Advanced  Technology  Development 

535 

Steve  Martin 

1992 

NCCOSC 

Advanced  Technology  Development 

535 

Walt  Aviles 

1993 

NCCOSC 

Adaptive  Systems 

531 

Walt  Aviles 

1994 

NCCOSC 

Adaptive  Systems 

531 

LCDR  Don  DeMuth 

1995 

NCCOSC 

Adaptive  Systems 

531 

LCDR  Don  DeMuth 

1996 

NCCOSC 

Adaptive  Systems 

D371 

John  Bott 

1997 

SSC  San  Diego 

Adaptive  Systems 

D371 

John  Bott 

1998 

SSC  San  Diego 

Adaptive  Systems 

D371 

John  Bott 

1999 

SSC  San  Diego 

Adaptive  Systems 

D371 

John  Bott 

2000 

SSC  San  Diego 

Adaptive  Systems 

D371 

Robin  Laird 

2001 

SSC  San  Diego 

Adaptive  Systems 

D371 

Robin  Laird 

2002 

SSC  San  Diego 

Robotic  Systems 

2371 

Robin  Laird 

2003 

SSC  San  Diego 

Robotic  Systems 

2371 

Robin  Laird 

2004 

SSC  San  Diego 

Unmanned  Systems 

2371 

Robin  Laird 

2005 

SSC  San  Diego 

Unmanned  Systems 

2371 

Hoa  Nguyen 

2006 

SSC  San  Diego 

Unmanned  Systems 

2371 

Hoa  Nguyen 

2007 

SSC  San  Diego 

Unmanned  Systems 

7171 

Hoa  Nguyen 

2008 

SSC  Pacific 

Unmanned  Systems 

7171 

Hoa  Nguyen 

2009 

SSC  Pacific 

Unmanned  Systems 

7171 

Hoa  Nguyen 

2010 

SSC  Pacific 

Unmanned  Systems 

7171 

Hoa  Nguyen 

2011 

SSC  Pacific 

Unmanned  Systems 

7171 

Hoa  Nguyen 

2012 

SSC  Pacific 

Unmanned  Systems  Group 

71710/20 

Hoa  Nguyen 

2013 

SSC  Pacific 

Unmanned  Systems  Group 

71710/20 

Tracy  Pastore/Rachel  TenWolde 

2014 

SSC  Pacific 

Unmanned  Systems  Group 

71710/20 

Tracy  Pastore/Rachel  TenWolde 

2015 

SSC  Pacific 

Unmanned  Systems  Group 

71710/20 

Tracy  Pastore/Rachel  TenWolde 

NOSC-  Naval  Ocean  Systems  Center 

NCCOSC  -  Naval  Command  Control  and  Ocean  Surveillance  Center 
SSC  San  Diego-  Space  and  Naval  Warfare  Systems  Center  San  Diego 
SSC  Pacific-  Space  and  Naval  Warfare  Systems  Center  Pacific 
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APPENDIX  F  -  PATENT  AWARDS  AND  APPLICATIONS 


PATENTS  AWARDED 

1.  Blackburn,  M.,  and  N.  Tran.  “Method  for  Vehicle  Collision  Avoidance,”  U.S.  Patent  #  8,175,796,  awarded  8 
May,  2012. 

2.  Bruch,  M.,  and  J.  Larson.  “Obstacle  Avoidance  System  and  Method,”  U.S.  Patent  #7,865,277,  awarded  4 
January,  2011. 

3.  Bruch,  M.,  and  J.  Larson.  “Method  for  Determining  Projected  Obstacle  Areas  for  Moving  Obstacles,”  U.S. 
Patent  #7,957,858,  awarded  7  June,  2011. 

4.  Bruch,  M.,  and  J.  Larson.  “Registration  of  Latitude/Longitude  Coordinates  Using  Range-Detection  Sensors  and 
Digital  Nautical  Charts,”  U.S.  Patent  #8,154,438,  awarded  10  April,  2012. 

5.  Burmeister,  A.  “System  and  Method  for  Measuring  an  Object’s  Center  of  Gravity,”  Navy  Case  #100,01 1,  U.S. 
Patent  #  8,229,701,  awarded  24  July,  2012. 

6.  Everett,  H.  R.,  and  C.  S.  Wright.  “Tactile  Bumper  for  a  Mobile  Robot  or  Platform,”  Navy  Case  #68,615,  U.S. 
Patent  #4,596,412,  awarded  24  June,  1986. 

7.  Everett,  H.  R.  “Programmable  Near-Infrared  Ranging  System,”  Navy  Case  #70,153,  U.S.  Patent  #4,851,661, 
awarded  25  July,  1989. 

8.  Everett,  H.  R.,  and  G.  A.  Gilbreath.  “Intelligent  Security  Assessment  System,”  Navy  Case  #71,393,  U.S.  Patent 
#4,857,912,  awarded  15  August,  1989. 

9.  Everett,  H.  R.  “Optical  Motion  Detector  Detecting  Visible  and  Near-Infrared  Light,”  Navy  Case  #71816,  U.S. 
Patent  #4,902,887,  awarded  20  February,  1990. 

10.  Everett,  H.  R.  “Pneumatic  Actuator  with  Hydraulic  Control,”  Navy  Case  #70,227,  U.S.  Patent  #5,058,385, 
awarded  22  October,  1991. 

11.  Everett,  H.  R.  “Reconfigurable  Video  Line  Digitizer,”  Navy  Case  #72,498,  U.S.  Patent  #5,034,817,  awarded  23 
July  1991. 

12.  Everett,  H.  R.  “Intelligent  Battery  Charging  System,”  Navy  Case  #70,154,  U.S.  Patent  #5,045,769,  awarded  3 
September,  1991. 

13.  Everett,  H.  R.  “Pneumatic  Actuator  with  Hydraulic  Control,”  Navy  Case  #70,227,  U.S.  Patent  #5,161,449, 
awarded  10  November,  1992. 

14.  Everett,  H.  R.,  G.  A.  Gilbreath,  R.T.  Laird.  “Navigational  Control  System  for  an  Autonomous  Vehicle,”  Navy 
Case  #72,770,  U.S.  Patent  #5,1 1 1,401,  awarded  5  May,  1992. 

15.  Everett,  H.  R.,  and  G,  A,  Gilbreath.  “Method  and  System  for  Fusing  Data  from  Fixed  and  Mobile  Security 
Sensors,”  Navy  Case  #72,775,  U.S.  Patent  #5,202,661,  awarded  13  April,  1993. 

16.  Everett,  H.  R.  “Doorway  Transit  Navigational  Referencing  System,”  Navy  Case  #73419,  U.S.  Patent 
#5,276,618,  awarded  4  January,  1994. 

17.  Everett,  H.R.,  and  G.  A.  Gilbreath.  “Reflexive  Teleoperated  Control  System  for  Remotely  Controlled  Vehicle,” 
Navy  Case  #72,949,  U.S.  Patent  #5,307,271,  awarded  26  April,  1994. 

18.  Everett,  H.  R.,  and  J.  M.  Nieusma.  “Feedback  System  for  Remotely  Operated  Vehicles,”  Navy  Case  #73,322, 
U.S.  Patent  #5,309,140,  awarded  3  May,  1994. 

19.  Everett,  H.R.,  and  R.  P.  Smurlo.  “System  for  Detecting  Perturbations  in  an  Environment  Using  Temporal 
Sensor  Data,”  Navy  Case  #75,144,  U.S.  Patent  #5,493,273,  awarded  20  February,  1996. 

20.  Everett,  H.R.,  R.  T.  Laird,  G.  A.  Gilbreath,  and  T.  A.  Pastore.  “System  for  Assigning  Computer  Resources  to 
Control  Multiple  Computer  Directed  Devices,”  Navy  Case  #76,192,  Patent  #5,659,779,  awarded  19  August, 
1997. 

21.  Everett,  H.R.,  G.  A.  Gilbreath,  R.  S.  Inderieden,  T.  T.  Tran,  and  J.  H.  Holland.  “Optically  Based  Position 
Location  System  for  an  Autonomous  Guided  Vehicle,”  Navy  Case  #77,796,  U.S.  Patent  #5,812,267,  awarded 
22  September,  1998. 

22.  Everett,  H.  R.,  and  L.  Albuquerque.  Scerri,  P.,  “Command  and  Control  System  for  Integrated  Human-Canine- 
Robot  Interaction,”  Navy  Case  #101,887,  U.S.  Patent  #  9,031,714,  awarded  12  May,  2015. 

23.  Gilbreath,  G.  A.,  D.  Powell,  and  D.  Barbour.  “System  and  Method  for  Displaying  Data  from  Multiple  Devices 
on  a  Single  User  Interface,”  U.S.  Patent  #9,030,491,  awarded  12  May,  2015. 
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24.  Holtz,  K.,  A.  Burmeister,  N.  Pezeshkian,  A.  Hart,  and  H.  Nguyen.  “System  and  Method  for  Remotely  Operated 
Deployment  and  Retrieval  of  Communication  Relays,”  Navy  Case  #101,287,  U.S.  Patent  #9,094,082,  awarded 
28  July,  2015. 

25.  Pezeshkian,  N.  “Close-Proximity  Communications  System  Using  Capacitively  Coupled  Signal  Transfer,”  Navy 
Case  #99,750,  U.S.  Patent  #8,396,136,  awarded  12  March,  2013. 

26.  Pezeshkian,  N.,  A.  Burmeister,  and  H.  Nguyen,  H.,  “Remotely  Operated  Illumination  Device,”  Navy  Case 
#100,338,  U.S.  Patent  #8,219,023,  awarded  10  July,  2012 

27.  Pezeshkian,  N.,  A.  Burmeister,  and  H.  Nguyen.  “Relay  Device  Deployer  System,”  U.S.  Patent  #8,103,212, 
awarded  24  January,  2012. 

28.  Pezeshkian,  N.,  A.  Burmeister,  H.  Nguyen,  K.  Holtz,  and  J.  Cruikshanks.  “Improved  Relay  Device  Deployer 
System,”  Navy  Case  #99,917,  U.S.  Patent  #8,103,212,  awarded  24  January,  2012. 

29.  Pezeshkian,  N.,  A.  Burmeister,  H.  Nguyen,  and  K.  Holtz.  “Next  Generation  Automatically  Deployed 
Communication  Relay,”  Navy  Case  #99526,  U.S.  Patent  #8,427,383,  awarded  23  April,  2013. 

30.  Pezeshkian,  N.,  A.  Burmeister.,  and  H.  G,  Nguyen.  “Wireless  Self-Contained  Relay  Device,”  U.S.  Patent 
#8,849,189,  awarded  30  September,  2014. 

31.  Pezeshkian,  N.,  A.  Burmeister,  and  H.  Nguyen.  “Wireless  Self-Contained  Relay  Device,”  Navy  Case  #98,480, 
U.S.  Patent  #8,849,189,  30  September,  20 14.54 

32.  Pezeshkian,  N.,  A.  Burmeister,  and  H.  Nguyen.  “Relay  Device  Deployer  System,”  Navy  Case  #98,975,  U.S. 
Patent  #8,909,130,  9  December,  2014. 

33.  Pezeshkian,  N.,  A.  Burmeister,  K.  Holtz,  A.  Hart,  H.  Nguyen,  and  L.  Roth.  “Method  of  Maintaining  an  Ad  Hoc 
Communications  Network  Between  a  Base  and  Mobile  Platform,”  Navy  Case  #102,533,  U.S.  Patent  9,107,081, 

1 1  August,  2015. 

34.  Pezeshkian,  N.,  J.  D.  Neff,  and  H.  R.  Everett.  “Adaptive  Electronic  Camouflage,”  Navy  Case  #101,118,  U.S. 
Patent  #9175930,  awarded  3  November,  2015. 

35.  Tran,  N.,  and  M.  Blackburn.  “Active  Composite  RFID  Tag  for  Object  Localization  and  Instruction,”  U.S.  Patent 
#8,094,012,  awarded  10  January,  2012. 

36.  Tran,  N.,  and  M.  Blackburn,  M.,  “Self-Contained  Sensor  Package  for  Water  Security  and  Safety,”  U.S.  Patent 
#8,134,462,  awarded  13  March,  2012. 

37.  Tran,  N.,  M.  Blackburn,  and  H.  Phan.  “Hall-Effect  Finger-Mounted  Computer  Input  Device,”  U.S.  Patent 
#8,246,462,  awarded  21  August,  2012. 

38.  Tran,  N.,  S.  Fugate,  L.  Duffy,  and  H.  Phan.  “RFID  System  for  Gesture  Recognition,  Information  Coding,  and 
Processing,”  U.S.  Patent  #8,279,091,  awarded  2  October,  2012. 

39.  Tran,  N.  “Hydraulic-Based  Spherical  Robot,”  U.S.  Patent  #  8,316,970,  awarded  27  November,  2012. 

40.  Tran,  N.,  S.  Fugate,  M.  Ceruti,  L.  Duffy,  H.  Phan.  “Hall-Effect  System  for  Gesture  Recognition,  Information 
Coding,  and  Processing,”  U.S.  Patent  #8,421,448,  awarded  16  April,  2013. 

41.  Tran,  N.,  M.  Bruch,  R.  Adam,  A.  Burmeister,  A.  Rahimi.  “Remotely  Controlled  Traffic  Management  System,” 
U.S.  Patent  #8,222,750,  awarded  15  April,  2013. 

42.  Tran,  N.,  M.  Bruch,  R,  Adam,  A.  Burmeister,  and  A.  Rahimi.  “Remotely  Controlled  Traffic  Management 
System,”  U.S.  Patent  #8,442,750,  awarded  14  May,  2013. 

43.  Tran,  N.,  H.  Phan,  T.  Ton,  J.  Rockway,  and  A.  Ton.  “Wand  Controller  for  Aircraft  Marshalling,”  U.S.  Patent 
#8,456,329,  awarded  4  June,  2013. 

44.  Tran,  N.,  M.  Bruch,  H,  Phan,  and  S.  Fugate.  “Active  Capacitive  Control  Stylus,”  U.S.  Patent  #8,648,837, 
awarded  11  February,  2014. 

45.  Tran,  N.  “RF-Based  System  for  Close-Proximity  Data  and  Energy  Transfer,”  U.S.  Patent  #8,660,491,  awarded 
25  February,  2014. 

3.  Burmeister,  A.  “Manually  Deployed  Communication  Relay,”  U.S.  Patent  #9094082,  awarded  . 

Applications  Submitted 

1.  Rosen,  G.,  B.  Chadwick,  B.  Nguyen,  A.  Burmeister,  and  H.  Nguyen.  “Surface  Sediment  Core  Catcher,”  Navy 
Case  #102,059,  filed  5  March,  2014. 

2.  Burmeister,  A.,  N.  Tran,  M.  Bruch,  R.  Halterman,  J.  Lum,  and  M.  Tjersland.  “Nodding  Mechanism  of  a  Single- 
Scan  Sensor,”  Navy  Case  #100,892,  filed  28  November,  2013. 


54  Received  the  “Michael  Kagan  2014  Invention  of  the  Year  Award.” 
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3.  Burmeister,  A.  “Manually  Deployed  Communication  Relay,”  Navy  Case  #101,287,  filed  23  May,  2012. 

4.  Holtz,  K.,  A.  Burmeister,  N.  Pezeshkian,  A.  Hart,  and  H.  Nguyen.  “System  and  Method  for  Remotely  Operated 
Deployment  and  Retrieval  of  Communication  Relays,”  Navy  Case  #101,207,  filed  14  May,  2012. 

5.  Larson,  J.,  .and  N.  Tran.  “Building  Inspection  and  Mapping  Device,”  Navy  Case  #101,320,  28  November,  2013. 

6.  Nguyen,  H.,  and  A.  Burmeister.  “VTOL  UAV  Landing  on  Uneven  or  Sloped  Terrain,”  Navy  Case  #102,573, 
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